Stop getting burned by Kubernetes ephemeral storage limit eviction

Technical Briefing | 8/26/2026

I have seen too many clusters go down because someone ran a build process inside a container that decided to write a few gigabytes of artifacts to local disk. You set your memory and CPU limits, you feel good about your manifest, and then the kubelet abruptly kills your pod because the emptyDir filled up the node. It is not always obvious that those scratch spaces are constrained by the same mechanism as your root filesystem.

Why your emptyDir isn’t as empty as you think

The kubelet monitors disk usage for container logs and emptyDir volumes using a dedicated process. When the combined usage hits your node’s eviction threshold, the node stops playing nice. It does not just kill the offending pod; it can mark the node as NotReady. The tricky part is that many admins forget that logs are also counted against this same bucket. If your application starts dumping massive stack traces to stdout, it triggers the same pressure events as a runaway data processing job.

du -sh /var/lib/kubelet/pods/*/volumes/kubernetes.io~empty-dir/*

• Ephemeral storage limits are applied to the sum of container writable layers, logs, and emptyDir volumes
• The kubelet check interval defaults to 10 seconds, which is plenty of time for a fast writer to trigger an OOM-like eviction
• Setting storage limits in your pod spec is the only way to isolate these bursts from the system partition

The reality of eviction metrics

Don’t rely on your cloud provider’s dashboard to tell you why the pod died if you see an Evicted status with a message about local storage pressure. Dig into the kubelet logs directly. If you are running a heavy-duty worker, consider using a separate partition for container runtime data or hostPath mounts if your security policy permits it. Otherwise, you are just waiting for a busy log rotation cycle to knock your production services offline.