Shell Scripting / Bash Tricks

Stop your nested loops from crawling at a snail’s pace

🧩 The Challenge

Dealing with a massive directory structure of logs is painful when you iterate using a standard while loop. You end up waiting forever just to process a few thousand files because the overhead is killing your script.

💡 The Fix

Switch to process substitution and xargs for your file processing needs. It keeps the CPU busy with the work you actually care about rather than spawning shell instances for every single line.

while read -r file; do
echo "Processing $file"
done < <(find /var/log -name "*.log") | xargs -P 4 -n 1 -I {} sh -c 'grep "ERROR" {} >> /tmp/errors.txt'

⚙️ Why It Works

Using process substitution avoids the subshell creation that usually happens with pipes and keeps the script context intact. By tossing xargs into the mix with the -P flag, you get true parallel execution which is a life saver when dealing with I/O heavy tasks.

🚀 Pro-Tip: Keep your -P count low if you’re hitting mechanical drives, otherwise you’ll just kill your own disk performance.

Linux Tips & Tricks | © ngelinux.com | 8/14/2026