Stop your OOM killer from being a blind executioner
By Saket Jain Published Linux/Unix
Stop your OOM killer from being a blind executioner
Technical Briefing | 7/22/2026
You set your memory limits in Kubernetes, you feel good about your resource planning, and then the OOM killer nukes your pod while the node metrics claim there was plenty of memory left. It is frustrating, silent, and usually happens at 3 AM. The kernel isn’t actually lying to you, but it is looking at a different map of reality than your Prometheus dashboards are.
The kernel does not care about your labels
The Linux kernel manages memory based on cgroups, but the OOM score calculation is a black box that prioritizes processes consuming the most memory relative to their allowed limits. When a container hits its hard limit, it’s not always the process that caused the allocation spike that gets killed. It’s often a sidecar, a logging agent, or a monitoring daemon that just happened to be the easiest target when the kernel started panicking. If you want to keep your critical processes alive, you have to talk to the kernel directly via the oom_score_adj file.
echo -1000 > /proc/self/oom_score_adj
- Use -1000 to essentially make a process unkillable by the OOM killer.
- Values between -999 and 1000 allow you to tune the survival priority.
- Remember that Kubernetes will overwrite this if you aren’t careful with your pod specifications.
Why your pod keeps dying despite low usage
Often the issue is page cache or buffer bloat that doesn’t show up as ‘RSS’ in your usual monitoring tools. The kernel sees memory pressure, triggers the reclaim cycle, fails to free enough space, and goes into defensive mode. If you see OOMKilled events without a corresponding spike in your application metrics, check the kernel ring buffer. Stop assuming it is your code leaking memory and start looking at what the kernel is doing under the hood to satisfy those page allocations.
Keep an eye on /proc/1/status in your containers to see the current adjustment score. If you find a pod that dies too frequently, check its score and adjust it before the next incident. You can’t stop the kernel from being a ruthless judge, but you can certainly influence the jury.
