You spend all afternoon crafting the perfect eBPF trace to catch a specific syscall. You attach it, your load averages instantly peg the CPU to a hundred percent, and the kernel hangs. It’s the classic observability trap: your probe is tracing the very function that handles the probe output, leading to a nasty recursive nightmare that usually ends with a hard reset.

Why the kernel hates your clever tracer

When you hook a kprobe, you are essentially telling the kernel to stop execution, run your code, then resume. If your helper function or your logging mechanism happens to call the same function you’re tracing, you’ve created a stack overflow in the middle of a kernel interrupt. I have seen this wipe out entire production clusters because someone wanted to trace write operations on a busy logging filesystem.

bpftrace -e 'kprobe:sys_write { @[comm] = count(); }' -v --filter 'comm != "bpftrace"'

• Filter your probes by PID or command name immediately to skip your own telemetry tools
• Avoid using printk inside hot path probes unless you have a death wish for your buffer
• Use kretprobes instead of kprobes if you only care about the result rather than the call itself

If you are testing new logic, run it inside a test VM first rather than your main monitor stack. It’s also worth using BPF helpers like bpf_get_current_pid_tgid to gate your execution early before you start logging data. Keep your logic lean and keep your probe surface area as small as possible, or you will eventually take down the machine you’re trying to monitor.