You probably have a CI/CD runner somewhere in your fleet that executes Ansible playbooks against a fleet of nodes. It works great until a job gets killed mid-flight because of a network hiccup or a timeout. The master runner dies, but the processes spawned on the remote node? They often hang out, locked in a zombie state or continuing to chew on resources until the next boot. I have seen this silently eat up CPU cycles on critical app servers for weeks.

Why the process tree gets orphaned

When Ansible executes a command via SSH, it spins up a shell on the target. If the connection drops or the parent process receives a SIGKILL, the sub-processes frequently ignore the hang-up signal if they were not explicitly started in a way that respects the parent-child signal chain. Most people just assume Ansible cleans up after itself. It tries, but once the underlying SSH transport layer goes dark, it loses visibility into those remote PIDs.

ansible all -m shell -a "nohup your-heavy-script.sh > /dev/null 2>&1 &" -B 3600 -P 0

• Use the -B flag to tell Ansible the job is asynchronous so it tracks the process ID
• Set -P 0 if you do not want Ansible to poll the status, which keeps the connection lean
• Always trap signals in your shell scripts so they exit gracefully when the parent dies

Cleaning up the mess you did not make

If you are already dealing with a pile of stranded processes, running a killall based on script name is dangerous if you have other instances running. Instead, check the process hierarchy and look for parents with a PID of 1. If your script is running as a child of init or systemd, it is a tell-tale sign that the original runner has already abandoned ship. Next time your automation starts feeling sluggish, check for these lingering ghosts before you go scaling up your infrastructure.