5 Linux Filesystem Utilities for Diagnostics

A great deal of the time, I work on the command line — usually logged into a remote system, doing some tasks or troubleshooting some problem. Quite often, this involves checking or manipulating something on the filesystem.

There are dozens of filesystem utilities. Most are well-known file manipulation utilities such as mv, rm, touch, mkdir, etc. However, there are several less familiar, but very powerful tools that I find myself using on a nearly daily basis. The following Linux filesystem utilities are ones I find particularly helpful for diagnosing issues and gathering information to solve problems.

Free Disk Space

Finding the amount of available free disk space is important — especially if a system has a low capacity hard drive or typically runs close to the margins. Whenever I start seeing strange failures on a system, one of the first things I check is disk utilization. The df command allows me to quickly check if a system is running near disk capacity.

Examples

Human-readable free disk space listing:

• df -h

  Filesystem            Size  Used Avail Use% Mounted on
  /dev/sda1              48G   41G  4.5G  91% /
  tmpfs                 1.9G     0  1.9G   0% /lib/init/rw
  udev                   10M  568K  9.5M   6% /dev
  tmpfs                 1.9G     0  1.9G   0% /dev/shm
  

Human-readable free inode listing:

• `df -hi`

  Filesystem            Inodes   IUsed   IFree IUse% Mounted on
  /dev/sda1            3115008  661565 2453443   22% /
  tmpfs                 220612       4  220608    1% /lib/init/rw
  udev                  220612    1076  219536    1% /dev
  tmpfs                 220612       1  220611    1% /dev/shm
  

Disk Space Utilization

Finding how much space certain files and directories take up is also important. If free disk space is unexpectedly low, I start searching for any files or directories which are larger than expected. The du command allows me to check how much space directories and files are utilizing. I start with the usual suspects such as /var/log and /var/lib, but will start at / to work my way through the entire disk if nothing obvious presents itself.

Examples

Human-readable summary of disk utilization of the current directory:

• cd /var/log
• du -sh
• => 103M

Human-readable summary of disk utilization of progressive directory lists:

• cd /
• du -sh *

  4.2M    bin
  15M     boot
  0       cdrom
  568K    dev
  4.9M    etc
  1.4G    home
  0       initrd.img
  67M     lib
  16K     lost+found
  12K     media
  4.0K    mnt
  2.4G    opt
  0       proc
  2.3G    root
  3.5M    sbin
  4.0K    selinux
  200K    srv
  0       sys
  108K    tmp
  985M    usr
  34G     var
  

• cd opt
• du -sh *

  2.2G    nginx
  0       ruby-enterprise
  145M    ruby-enterprise-1.8.7-20090928
  

• cd nginx
• du -sh *

  4.0K    client_body_temp
  40K     conf
  4.0K    fastcgi_temp
  12K     html
  2.2G    logs
  4.0K    proxy_temp
  7.7M    sbin
  

Aha! The logs.

File Type

File extensions are nice hints, but they aren’t always correct. It’s nice to know the probable contents of a file before trying to cat it. I’ve rendered many a terminal session unusable by cating binary files which I thought were text. The file utility will examine the contents of a file and try to identify the data type, allowing for an educated guess of which program to use.

Examples

Detecting data type of a file:

• file tada
• => tada: GIF image data, version 89a, 350 x 263

Output MIME type of a file:

• file -i regular-1.ttf
• => regular-1.ttf: application/x-font-ttf; charset=binary

Link Value / Canonical Name

Mazes of symlinks and deeply-nested directories can cause confusion. Are two given paths really referencing the same file? What is the absolute path to this file nested 10 directories deep? Fortunately, the readlink utility can help straighten this all out. The read link utility can find the target for a symlink, determine the canonical path to the target of a symlink, or just return the absolute path to a file.

Examples

Getting the target of a symlink:

• cd /var/www/html
• readlink current
• => releases/20130508130750/

Getting the canonical path of symlink:

• cd /var/www/html
• readlink -f current
• => /var/www/html/releases/20130508130750/

Getting the absolute path of a file:

• cd /var/www/html/current/log
• readlink -f production.log
• => /var/www/html/shared/log/production.log

Output Files as Hex (or Decimal, or Octal)

The encoding or data format of text files can prove to be an interesting challenge. Given a UTF-16 file, the results you get from cat and vim could change dramatically depending on whether the file includes a byte-order mark (BOM), or if the endianess matches (or doesn’t match) the endianess of your system. Fortunately, od will allow you to dump the raw data in various formats so that you can figure things out.

Examples

Hexadecimal dump of a UTF-16 file with BOM:

• od -x utf16

  0000000 fffe 5400 6800 6900 7300 2000 6600 6900
  0000020 6c00 6500 2000 6900 7300 2000 6500 6e00
  0000040 6300 6f00 6400 6500 6400 2000 6900 6e00
  0000060 2000 5500 5400 4600 2d00 3100 3600 2000
  0000100 7700 6900 7400 6800 2000 6100 2000 4200
  0000120 4f00 4d00 2e00 0a00
  0000130
  

Hexadecimal dump of a UTF-16 file without BOM:

• od -x utf16le

  0000000 0054 0068 0069 0073 0020 0066 0069 006c
  0000020 0065 0020 0069 0073 0020 0065 006e 0063
  0000040 006f 0064 0065 0064 0020 0069 006e 0020
  0000060 0055 0054 0046 002d 0031 0036 004c 0045
  0000100 002e 000a
  0000104
  

Conclusion

While certainly not a comprehensive collection, df, du, file, readlink, and od rank highly in my Linux filesystem toolbox. They are available in some form on most UNIX-like systems, including BSD and OSX (though the functionality of some, such as readlink may be somewhat reduced). Given how often I use them, I’ve installed the GNU coreutils versions of these utilities on my Mac so I can use the familiar functionality and flags present in the Linux variants. What Linux filesystem utilities do you find most helpful?