Find Hidden Linux Disk Usage with df, du, and lsof

A Linux filesystem can report 95% usage in df while du appears to account for much less. The tools are not contradicting each other: they measure different things.

df asks the filesystem about allocated blocks. du walks visible directory entries and sums blocks reachable through those paths. The gap between those views points to several useful troubleshooting cases.

Start with the filesystem view

Check filesystems and their types:

df -hT

Identify the mount that is actually full. Do not immediately scan recursively from /; container mounts, network filesystems, and bind mounts can make that slow and misleading.

Also check inode usage:

df -i

A filesystem can run out of inodes while still having free data blocks, especially when it contains millions of tiny files.

Measure visible directory usage

For a local filesystem mounted at /var, start one level deep:

sudo du -xhd1 /var | sort -h

Then descend into the largest directory:

sudo du -xhd1 /var/lib | sort -h

The -x flag keeps du on one filesystem, which is important when diagnosing one specific mount.

When df and du disagree

A large discrepancy often means blocks are allocated to files that no longer have directory entries.

On Unix-like filesystems, deleting a file removes its name, but the data remains allocated while a process still has the file open. Logging processes are a common example.

Find deleted-but-open files with:

sudo lsof +L1

Look for large entries marked as deleted. The owning process can continue writing to the invisible file, so disk usage may keep growing.

Recover space safely

The safest fix is normally to make the application reopen its files using its documented reload or restart mechanism.

For a systemd-managed service, that may involve:

sudo systemctl restart example.service

Do not restart production services blindly. Confirm the process, redundancy model, and operational impact first.

Avoid truncating arbitrary /proc/<pid>/fd/<n> paths unless you fully understand the application’s file semantics.

Check for hidden data under mount points

Another mismatch can happen when files were written into a directory before another filesystem was mounted over it.

Suppose /data contained 20 GB on the root filesystem, then a separate volume was mounted at /data. The old files still consume root filesystem blocks but are hidden by the mount.

Inspect mounts with:

findmnt

Resolving covered data safely normally requires a maintenance path where the covering filesystem can be unmounted or the underlying root filesystem inspected elsewhere.

Understand sparse files

A sparse file may have a large logical size while consuming far fewer physical blocks.

Compare logical and allocated sizes:

ls -lh large-file
du -h large-file
du --apparent-size -h large-file

This is different from a deleted-open-file problem: a sparse file looks huge by length but may consume little storage.

Container and overlay filesystems

On container hosts, disk pressure may live in overlay layers, image caches, writable container layers, or logs.

First identify the mount reported by df, then use the container runtime’s supported storage commands. Do not delete files directly from runtime-managed storage directories while the runtime is active.

The same principle applies to databases and package managers: prefer their supported cleanup mechanisms over manual deletion.

A repeatable diagnostic sequence

When a filesystem is unexpectedly full:

1. run df -hT and identify the exact mount;
2. run df -i to rule out inode exhaustion;
3. use du -xhd1 to locate visible usage;
4. run lsof +L1 if du cannot explain df;
5. inspect findmnt for covered directories or unexpected mounts;
6. consider filesystem-specific reserved space;
7. use application-aware cleanup rather than deleting unknown files.

Common pitfalls

Deleting active log files

Removing a log path does not guarantee immediate space recovery if a process still holds the old inode.

Crossing filesystem boundaries with du

Without -x, a scan may include mounted volumes unrelated to the full filesystem.

Treating every large file as disposable

Large database, journal, or package files may be essential. Identify ownership before cleanup.

Conclusion

df and du answer different questions. When their numbers diverge, deleted open files, covered mount points, inode pressure, sparse files, and filesystem reservations are the main places to investigate. A structured diagnosis is safer than deleting whatever looks large.