Stop the OOM Killer from making up its own mind about your memory limits

Technical Briefing | 9/3/2026

You set your container memory limits to 2Gi, the app spikes, and the kernel sends a SIGKILL. That is standard behavior. But I have seen countless production environments where containers hit memory limits far below their configured cgroup caps, or worse, stay alive until the host node becomes unresponsive. It turns out, if you rely strictly on Kubernetes manifests without digging into the underlying memory controller stats, you are flying blind.

Why the kernel ignores your manifest

The kernel doesn’t care about your YAML files. It cares about the memory.stat file inside the cgroup hierarchy. When your application performs extensive file I/O or shares libraries, the kernel creates page caches that count toward your cgroup limit. If your memory usage is primarily cache rather than resident set size, your process is a prime candidate for an eviction, even if the application feels like it has plenty of headroom. You need to verify what is actually hitting the limit before you blindly increase the request or limit.

cat /sys/fs/cgroup/memory/kubepods.slice/kubepods-pod<pod_id>.slice/docker-<container_id>.scope/memory.stat

• Check cache vs anon memory usage to see if file I/O is stealing your RAM
• Use rss_huge to see if transparent hugepages are inflating your footprint without warning
• Monitor failcnt to see if your container is hitting its hard limit frequently

What to do when you hit the ceiling

If you are consistently seeing high cache usage, tweaking your kernel’s swappiness or investigating page-cache-heavy dependencies is a better path than just throwing more memory at the pod. And if you are still losing containers randomly, check the dmesg logs for oom-kill events specifically tied to the cgroup hierarchy. Most of the time, the solution is not more RAM, but better control over how your app interacts with the filesystem buffers. Next time a pod drops, skip the kubectl events and check the host kernel logs first.