Backup & Recovery (Rsync/Tar/Dd)
Stop letting rsync crawl through your massive file lists
🧩 The Challenge
Dealing with a backup directory containing millions of tiny files is a special kind of hell because rsync spends ten minutes just walking the tree before moving a single byte. You’re left staring at a blinking cursor wondering if the whole thing hung or if it’s just doing its tedious homework.
đź’ˇ The Fix
Throw the –files-from flag into the mix to skip the recursive scan entirely. It forces rsync to take a literal list of files you provide instead of burning clock cycles traversing your filesystem.
find /source/dir -type f -mtime -1 > /tmp/to_backup.txt
rsync -avz --files-from=/tmp/to_backup.txt /source/dir/ /destination/dir/⚙️ Why It Works
Passing an explicit list lets you offload the heavy lifting to find, which is significantly faster at chewing through massive directory structures than the built-in rsync walker. Doing this also prevents rsync from trying to sync every single file if you only need the ones that changed in the last day.
🚀 Pro-Tip: Pipe that find output through grep or sort if you need to be surgical about what actually makes the cut.
Linux Tips & Tricks | © ngelinux.com | 10/17/2026