Keep Your CI Runners from Starving Each Other with Systemd Slice Units

Technical Briefing | 10/18/2026

You spin up a new CI runner, point it at your build farm, and things are humming along. For a while. Then you start seeing builds mysteriously fail with “out of memory” errors, or suddenly take twice as long. Sometimes, it’s just the entire server bogging down, even if top shows no single culprit pegging the CPU. It’s frustrating, and I’ve seen it quietly eat away at productivity on shared infrastructure more times than I care to count. This usually happens when you’ve got multiple pipelines kicking off jobs on the same host, all happily competing for finite resources.

The Silent Killers: When Builders Compete

Here’s the thing: most of us are good about running CI jobs inside containers. That’s a given for isolation, right? But the container runtime itself, along with any helper processes your runner daemon spawns, still runs directly on the host. And if you have three containers simultaneously trying to compile a massive C++ codebase, or fetch huge Git repos, they’re all drawing on the host’s RAM, CPU, and I/O. Without some form of host-level resource governance, it’s a free-for-all. A single greedy build can starve out everything else, leading to cascading failures or just plain terrible performance.

Systemd Slices: Your Resource Guardrails

This is where systemd slice units become your best friend. They’re a direct, elegant way to interact with Linux control groups (cgroups), letting you group processes and apply limits to them. Think of them as containers for your processes on the host, managing resource allocation for the kernel. You can create a hierarchical structure, too. Put all your CI runner services under one parent slice, then specific runner instances under sub-slices, each with its own limits. This granularity is powerful, and most tutorials skip right over it.

/etc/systemd/system/ci-runners.slice
[Unit]
Description=Resource-limited slice for CI build processes


[Slice]
CPUQuota=200%
MemoryLimit=6G
IOReadBandwidthMax=/dev/sda 100M
IOWriteBandwidthMax=/dev/sda 100M


/etc/systemd/system/gitlab-runner.service.d/override.conf
[Service]
Slice=ci-runners.slice
Delegate=yes


sudo systemctl daemon-reload
sudo systemctl restart gitlab-runner
sudo systemctl status ci-runners.slice

A quick explanation of those options: CPUQuota sets a percentage of total CPU time available to processes in the slice. 200% means two full CPU cores. MemoryLimit is straightforward: hard limit on RAM. And IOReadBandwidthMax/IOWriteBandwidthMax let you cap disk I/O, which is huge for noisy neighbors. That /dev/sda bit is important – you’re specifying the device. Delegate=yes is also key; it tells systemd to let the processes within the slice create their own nested cgroups, which is exactly what container runtimes like Docker or Podman do. Without it, you might find your container’s resource limits being ignored by the parent slice.

• Don’t over-allocate: If you have 8 CPU cores, CPUQuota=800% is your max. Setting it to 100% means one core.
• Monitor actual usage: Look at systemd-cgls, systemd-cgtop, or /sys/fs/cgroup/… to see what’s actually happening before you set hard limits.
• Prioritize critical jobs: You can create multiple slices with different priorities if some builds are more important.
• Test rigorously: A too-tight limit can kill builds just as effectively as a resource free-for-all. Start loose, then tighten.

This approach puts you in charge of your host’s resource landscape. It’s a bit more effort than just docker run, but when you’re managing self-hosted runners that are shared across teams, or running critical deployments, this level of control is absolutely worth it. You’re not just hoping the kernel does the right thing; you’re telling it exactly what to do. And that’s how you keep your build times predictable and your systems stable, even when everyone decides to run their largest pipeline at 2 PM on a Friday.