• “No space left on device” Can Mean “No Inodes Left”

    Sometimes Linux says No space left on device while df -h insists that the disk has free space. That looks contradictory, but it usually means you are looking at the wrong resource.

    df -h reports block usage: how many bytes are used and free. A filesystem also has to track files themselves. On traditional Unix-like filesystems that metadata lives in inodes. If the filesystem runs out of available inodes, it may be unable to create a new file even when there are gigabytes of block space left.

    I first hit this through a PHP session error:

    PHP Warning: Unknown: open(/tmp/sess_e34ad6u6f51gum3htmqkd7ldn6, O_RDWR) failed:
No space left on device (28)

    The confusing part was that df -h looked fine:

    df -h

Filesystem          1K-blocks     Used Available Use% Mounted on
/dev/sda2            19091584  3784332  14314332  21% /
udev                  8192628        8   8192620   1% /dev
tmpfs                 3280992      284   3280708   1% /run
none                     5120        4      5116   1% /run/lock
none                  8202472       92   8202380   1% /run/shm
/dev/mapper/vg0-var  47926152 24100456  21368096  54% /var

    The next thing to check is inode usage:

    df -i

    or, more explicitly:

    df --inodes

    GNU Coreutils documents df --inodes as the mode that reports inode information instead of block usage. The output includes total inodes, used inodes, free inodes, and inode usage percentage.

    Why It Happens

    An inode is not a file descriptor. A file descriptor is a per-process handle to an open file. An inode is filesystem metadata for a file-like object: type, ownership, permissions, timestamps, link count, and pointers to storage. The Linux man-pages describe this metadata in inode(7).

    Running out of inodes usually means the filesystem contains too many small files. Common causes:

    • PHP session files piling up in session.save_path
    • application cache directories with millions of tiny entries
    • temporary upload chunks or generated thumbnails
    • mail queues, package manager caches, or failed job artifacts
    • test suites creating fixtures and not cleaning them up

    PHP is a common place to notice the problem because the default file-based session handler writes session data into files. When PHP cannot create or open the session file, the surface error may still be the generic ENOSPC: no space left on device.

    Find the Directory That Eats Inodes

    Start with the affected mount:

    df -i /tmp /var /var/lib/php

    Then look for directories with suspiciously high file counts. This can be expensive on a large filesystem, so run it on the affected mount, not blindly on /:

    sudo find /tmp -xdev -type f -printf '%h\n' |
  sort |
  uniq -c |
  sort -nr |
  head -20

    For PHP sessions, also check where sessions are stored:

    php -i | grep 'session.save_path'

    The PHP manual documents session_save_path() and the session.save_path setting; the exact location depends on your distribution and PHP-FPM/CLI configuration.

    Clean Up Carefully

    Do not fix this with a dramatic rm -rf /tmp/* unless you enjoy surprise downtime. Find the owner of the files and clean the specific cache, session, queue, or temporary directory.

    For old PHP session files, the correct fix is usually one of these:

    • restore the distribution’s session cleanup job;
    • fix PHP session garbage collection settings;
    • move sessions to a dedicated directory or backend;
    • reduce application behavior that creates excessive session files.

    For application caches, prefer the application’s own cleanup command. For queues and generated artifacts, delete only files that are safe to recreate.

    Prevent It

    Monitor both block usage and inode usage. Disk alerts that only watch df -h miss this failure mode.

    If a service legitimately creates huge numbers of tiny files, consider isolating that workload on its own filesystem, using a storage backend designed for it, or formatting the filesystem with an inode density that matches the workload. The best fix is usually architectural: fewer tiny files in a hot shared filesystem.

    No space left on device is not always about bytes. Sometimes the filesystem has room for data, but no room for another name in the catalog.