You spend all morning chasing a latency spike that only hits under heavy traffic. Your CPU usage looks fine at first glance, but the p99s are going off a cliff. It’s not the application code, and it’s not the disk. It’s the kernel struggling to keep up with packets because all your hardware interrupts are pinned to core zero. That’s the default behavior, and it’s almost always wrong for modern high-speed NICs.

Why core zero is a trap

Most of the time, the kernel defaults to assigning interrupts to the first CPU core. If your traffic is heavy, core zero becomes a massive bottleneck for softirqs. You’ll see high system time and dropped packets in your netstats, even when the rest of your cores are basically idling. This bit me hard on a web server last year; the load balancer reported healthy metrics, but the end-users were timing out.

grep . /proc/irq/*/smp_affinity_list

• Check your current IRQ balance to see if core zero is doing all the heavy lifting
• Enable irqbalance or manually map interrupts to cores near the NUMA node of your NIC
• Verify your NIC driver actually supports multi-queue hardware filtering

Fixing the affinity without breaking things

Don’t just blindly move everything away from zero. You need to verify your hardware supports multiple queues first. If you try to force affinity on a card that doesn’t support it, you might end up with kernel errors that are a nightmare to trace. Always check the queue count in ethtool before you start messing with the affinity masks.

Next time the network team complains about mystery jitters, ignore the app metrics for a minute. Pull up your interrupt stats and see if one core is suffering while the others sleep. It’s a quick fix that saves you hours of digging through application logs that weren’t the problem in the first place.