One of the most confusing failures in Linux: applications crash with No space left on device (ENOSPC), yet df -h swears the disk is only half full. The disk isn't out of bytes — it's out of inodes, the fixed-size table that tracks every file and directory on the filesystem.
Why it happens
An ext4 filesystem allocates its inode count at format time. Create millions of tiny files — a runaway mail spool, session files, a cache that never expires — and you can exhaust the inode table long before you fill the disk. Every new file needs an inode; when there are none left, writes fail even with gigabytes free.
Confirm it with df -i
df -h shows bytes; df -i shows inodes. If IUse% is at 100% while Use% looks fine, you've found it:
df -i # IUse% at 100% = inode exhaustion # find the directory holding the most files: for d in /var/spool/* /tmp; do echo "$(find "$d" | wc -l) $d"; done | sort -rn | head
Reclaim the inodes
Delete or consolidate the swarm of tiny files. For a spool directory full of stale entries: find /var/spool/clientmqueue -type f -delete. Then stop it recurring — a systemd-tmpfiles rule in /etc/tmpfiles.d/ will age the directory out automatically so you don't fight the same fire next week.
Practise the find-and-fix loop
Inode exhaustion is one of the incidents the Chaos Engine throws at you — it looks exactly like a full disk until you run df -i. Drill it in Linux troubleshooting practice, and compare with its byte-based cousin in “No space left on device”.