#How to Use lsof in Linux (With a Practical Example) – CloudSavvy IT

Ever wondered how to discover what files are currently open and in use on your system? The Linux lsof command lists open files and provides plenty of extra information. Learn how to use lsof with these practical examples.

What Is lsof?

Available natively within any Linux operating system, the lsof command provides a list of open files. However, the output can be a bit cryptic and long, especially when using many applications on a given system. Let’s have a look at basic lsof output. We will be running lsof as root.

To login as root, you can open a terminal window and type a command like sudo su or su to receive a root authentication prompt. Alternatively, you can execute sudo lsof. Note that while you can run lsof as a normal non-root user also, you may find that the output is incomplete.

Example: Basic lsof Output

sudo su
lsof | head -n10

Here we started the lsof tool using the lsof command, and captured the first ten lines of the output using a pipe (|) to sent the information output by lsof to a secondary head -n10 command which captures only the top (head) ten lines.

The lsof command lists various columns. First up we see the COMMAND column which lists the binary which is holding the open file/open file lock. Next we see the process ID column PID which is extremely handy when it comes to debugging various application issues, including incorrectly held file locks. In the next section we will look at an example of how we can use lsof in combination with kill to (destructively) terminate applications which hold incorrect file locks.

If supported on your operating system, the TID column can help to realize whether a given line is a process or a task. If the output is blank, as can be seen in our example (which is running on Linux Mint, an operating system that supports the TID column output), the given line/command is a process, a non-task. If you, for example, start a calculator app in your operating system, i.e., a task, this column will be populated by a task/thread identification number.

The TASKCMD column holds the task command name. Again it is only visible if the given line is a task and not a process. This is regularly the same as the command/process shown in the first COMMAND column, though, for example, Linux allows a task to change its command name, so it could contain additional information about the task. The USER column lists the user who started the process/task.

Next up we have an important column FD (File Descriptor) which can list the File Descriptor Number or a specific text string indicating what type of file descriptor this is. For example, if you see cwd in this column, it stands for current working directory and it indicates that the given process or task is holding an open lock on the current working directory and that working directory is listed under the NAME column.

For a full list of all possible FD strings, type man lsof at your command prompt followed by typing / FD (with 3 spaces after slash and before FD) once inside the manual, and then pressing ENTER to search for the FD section.

When it comes to regular files, you may think about the FD column like a counter, or a unique ID counter, starting at 0 and increasing to the total number of regular open files on the system, with the maximum number of open files being defined by the ulimit -n setting, etc.

When looking at regular files, the FD column will for example show 102u or 13w. These two examples respectively represent the 102th open file on the system, in mixed read/write access mode, as indicated by the u indicator/label and the 13th open file on the system, in write access mode only.

The TYPE column is quite self-explanatory; it indicates whether a regular file or a directory open lock is being held. There are various other labels that could be displayed here, and a search for / TYPE in man lsof (as explained above) will provide a full list.

The DEVICE column generally lists the device numbers, separated by commas, for most types of files. The SIZE/OFF column is the size of the file, or the file offset in bytes, and the NODE column commonly displays the file node or NFS file inode number of local files. It can also list for example TCP or UDP when the given lock is an open Internet connection.

Example: Using lsof with kill

Using lsof in combination with grep, awk and kill one can search a given system for a specific file and subsequently terminating the process (using the PID, or Process Identifier, described earlier).

Note that this should only be done in situations where it makes sense to do so. For example, you may have a multi-threaded Bash script that starts many different subshells where each one holds on to a given file, and you expect that one of the subprocesses is hanging. You know the open filename of the hanging process but do not know what the PID for that process/task is.

In such a situation, we can execute the following commands:

sudo su
lsof | grep 'some_file_descriptor' | awk '{print $2}' | xargs -I{} kill -9 {}

Let’s put this into a practical example. Assume we have created the following script test.sh in /tmp:

rm -Rf workspace
mkdir workspace
cd workspace
sleep 36000

This script, when executed, will create a directory named workspace, change directories into it with cd and then sleep for 10 hours. Whilst at first glance this would not seem to open any files, remember the cwd discussed earlier; the script has a current working directory in use namely /tmp/workspace!

Let’s see how this works out practically. First, we define (using your favorite text editor like vi), and then start the script in one terminal session:

Next we jump into a secondary/new terminal session and execute lsof, looking for the script’s work directory namely workspace:

As we can see, lsof is able to find not only our test script test.sh which is holding a cwd lock on /tmp/workspace, but we also see that sleep, started from within the script, is holding a cwd open file (or better directory) descriptor on the same directory.

We can also see both the PID for the test.sh script as well as for the sleep command. Let’s assume for a second that our test.sh script was hanging, and we wanted to completely terminate it destructively, i.e. with kill -9 which is the most destructive way to terminate a process without any level of graceful shutdown, and this based on any open files (or in our case open directories) the script was holding. We issue the following command from the secondary terminal:

lsof | grep "workspace" | awk '{print $2}' | xargs -I{} kill -9 {}

This command will take the previous output and parse it further. The awk '{print $2}' basically samples out the secondary column (the process ID’s) and the xargs command will pass this second column to kill -9 (the {} in the command will be replaced by whatever input xargs receives).

We could also anticipate that lsof has a header line that contains PID and this text will be passed onto kill as well. Whilst it will not create an issue, a better overall command would have read lsof | grep "workspace" | grep -v "PID" | awk '{print $2}' | xargs -I{} kill -9 {}, or some other way of filtering the first line altogether.

The result in our first/primary terminal session is that our script is instantly killed:

If you would like to learn more about xargs, you can read about Using xargs in Combination With bash -c to Create Complex Commands. For Bash multi-threaded programming, see How to Use Multi-Threaded Processing in Bash Scripts.

Wrapping up

In this article we explored using lsof, the list open files command, and what information each of the columns in its default output represents. We also looked at a practical use case where we used lsof in combination with grep, awk and kill to find a particular file (or in our case a directory) being held open by a script and subsequently terminate that script thereby closing the open directory.

If you enjoyed reading this article, have a look at our Asserts, Errors, and Crashes: What’s the Difference? article.