Stop the OOM Killer from hiding its victims in your cgroup v2 hierarchy

Container & Kubernetes Internals

Stop the OOM Killer from hiding its victims in your cgroup v2 hierarchy

Technical Briefing | 11/16/2026

You see your pod restart with an Exit Code 137 and assume the container just hit its memory limit. Usually, that is exactly what happened. But sometimes, when you check the logs, it feels like the process was nuked out of thin air without a clear trail. In the shift to cgroup v2, the kernel’s OOM behavior changed, and the legacy logs you relied on have shifted too.

The kernel is no longer shouting into dmesg

On older systems, the OOM killer was noisy. It would dump a full process list into the kernel ring buffer, letting you see exactly who was causing memory pressure. Under cgroup v2, if a container hits its hard limit, the kernel is more restrained. It often just kills the process without filling your log files with a massive memory report. It is cleaner, but it makes debugging these silent evictions a nightmare if you are not looking in the right place.

cat /sys/fs/cgroup/memory.events

  • high: Number of times memory usage went above the high boundary
  • max: Number of times memory usage was throttled to remain under the hard limit
  • oom: Number of times a process was killed because the cgroup exceeded its limit
  • oom_kill: Number of processes killed within the cgroup by the OOM killer

Where to look when the pod just vanishes

If you are trying to catch this in the act, stop grepping the system log for OOM. Instead, look at the memory.events file inside the specific cgroup directory for the failing container. It is a simple counter. If oom_kill is incrementing, you have a hard limit breach. If oom is 0 but you still see evictions, you might be dealing with memory pressure leading to direct reclaim. Keep an eye on those counters; they are the only truth you have when the kernel goes quiet.

Next time a container drops off the map, jump into the host and inspect the cgroup v2 tree before you start guessing. Knowing exactly which event counter flipped will save you from chasing ghosts in your application code when the issue is just an aggressive limit.

Linux Admin Automation  |  © www.ngelinux.com  |  11/16/2026

0 0 votes
Article Rating
Subscribe
Notify of
guest

0 Comments
Newest
Oldest Most Voted