Site icon New Generation Enterprise Linux

Stop your Kubernetes OOMKiller from lying about who actually ate the memory

Container & Kubernetes Internals

Stop your Kubernetes OOMKiller from lying about who actually ate the memory

Technical Briefing | 8/3/2026

You see an OOMKill event in your Kubernetes dashboard and naturally head straight for the logs of the pod that died. You find nothing. The application didn’t crash; the kernel just nuked it to save the node. It is maddening because the memory metrics in your monitoring tool show usage well below the pod limits. You start questioning if your monitoring agent is broken or if the kernel is hallucinating.

The kernel doesnt care about your container limits

The fundamental issue is that the OOMKiller looks at the entire node’s memory pressure, not just your container’s cgroup limit. When the physical host starts starving, the kernel invokes the OOM notifier, and it looks for the process with the highest oom_score. This score is a heuristic mix of memory usage and longevity, and sometimes your most stable service becomes the easiest target.

cat /proc/$(pgrep -u appuser)/oom_score
  • The oom_score_adj file allows you to protect critical sidecars from being picked
  • Kernel-level memory pressure stall information is what actually triggers the eviction
  • Total node memory and container limits are distinct subsystems in the kernel

If you are running workloads with high churn or bursty memory patterns, check the node-level events instead of the pod metrics. If the node runs dry, Kubernetes will kill whatever process it deems least significant to the overall stability. Stop hunting for leaks in your code when the real culprit is a neighbor container that is slowly ballooning and triggering the node-wide pressure alarm.

Linux Admin Automation  |  © www.ngelinux.com  |  8/3/2026
0 0 votes
Article Rating
Exit mobile version