Stop Systemd from silently nuking your long-running processes

Systemd & Service Management

Stop Systemd from silently nuking your long-running processes

Technical Briefing | 7/13/2026

You spend all afternoon tuning a worker process to crunch through a massive dataset, only to find the unit dead after an hour. Checking the logs, you see it didn’t crash. Instead, systemd simply decided it had reached its lifespan or hung, and sent a SIGTERM. It is infuriating, but usually, it is your own configuration biting back.

The silent watchdog timer trap

Most of us ignore the WatchdogSec directive because it sounds like a fancy way to handle crashes. It is actually a heart-beat mechanism. If your application doesn’t notify systemd that it is still alive, the service manager assumes the process is deadlocked and kills it. If your task performs blocking I/O for longer than the watchdog interval, you are going to get slaughtered every time.

systemctl show your-service-name --property=WatchdogUSec

  • Check if WatchdogSec is set to anything other than 0 in your service file
  • Verify your application code actually implements the sd_notify heartbeats correctly
  • Consider extending RuntimeMaxSec if your process is legitimately hitting a forced expiry

If you don’t actually need systemd to watch your process’s internal health, make sure WatchdogSec is explicitly commented out or set to zero. Don’t let default unit templates add guardrails you didn’t ask for. Next time your service vanishes, check the exit code in journalctl first. If you see code 143, that is just the kernel’s way of saying your service manager lost patience.

Linux Admin Automation  |  © www.ngelinux.com  |  7/13/2026

0 0 votes
Article Rating
Subscribe
Notify of
guest

0 Comments
Newest
Oldest Most Voted