Stop trusting your snapshots to be actual backups

Technical Briefing | 7/24/2026

You set up a ZFS or LVM snapshot schedule, piped it to a remote host, and slept well knowing your data was covered. Then the disk controller on your primary array threw a bit-flip, and suddenly those perfectly replicated snapshots contain a corrupted filesystem. It is the classic sysadmin trap: snapshots are not a disaster recovery strategy, they are just a local undo button.

Why atomic doesn’t always mean safe

A snapshot only captures the state of the block device at a moment in time. If the application writing to that disk hasn’t flushed its memory buffers to the platter, the snapshot is essentially a crash-consistent mess. You think you have a clean state, but when you mount that snapshot later, the database engine spits out recovery errors because it can’t reconcile the half-written WAL segments. I have wasted entire weekends trying to reconstruct logs from these supposedly bulletproof snapshots.

zfs send -R pool/data@snap1 | ssh backup-host zfs recv -F pool/backups/data

• Use application-level hooks to quiesce databases before triggering the snapshot
• Verify checksums on the remote side if you are shipping data across a network
• Test a restore from your secondary site at least once a quarter

The hidden cost of stale snapshots

Disk latency is the silent killer when you keep too many snapshots. Every time you write a block, the copy-on-write overhead grows as the pointer tree gets deeper. If you’ve been running snapshots every hour for six months, you are essentially dragging an anchor through your I/O stack. You’ll notice it when disk utilization hits 80 percent, and the entire system slows to a crawl despite the CPU idling.

Don’t just automate the creation of these things and walk away. If you aren’t periodically mounting them to run consistency checks, you aren’t performing backups, you are just delaying the inevitable discovery that your data is gone. Make it a cron job to verify, not just a job to store.