CRITICALDockerContainers

Docker builds failing with 'no space left on device'

dockerdiskprunecibuildkit

Symptoms

docker build fails with 'write /var/lib/docker/tmp: no space left on device'
df -h shows /var/lib/docker at 100%
CI jobs start failing intermittently

Root Cause

Accumulated dangling images, stopped containers and build cache
Large layers in multi-stage builds retained by BuildKit
Overlay2 driver writing to a volume without an auto-prune policy

Diagnosis

docker system df to see disk usage by type
du -sh /var/lib/docker/* | sort -h
journalctl -u docker for any driver errors

Fix

Reclaim space immediately:

docker system prune -af --volumes
docker builder prune -af

Move Docker root to a larger disk if needed:

{
  "data-root": "/mnt/docker"
}

Restart the daemon: `systemctl restart docker`

Prevention

Schedule a nightly `docker system prune -af --filter until=72h`
Use `--mount=type=cache` in Dockerfiles instead of COPY caches
Monitor disk with a 75% alert threshold