When your block-level backups are lying to you

Technical Briefing | 7/13/2026

You set up LVM snapshots or ZFS send streams, verified the transfer, and patted yourself on the back. But then you tried to restore that database dump from last Tuesday and the file header was garbage. It happens because we often conflate the existence of a snapshot with the integrity of the data within it. Just because the block device was captured doesn’t mean your filesystem or application was in a sane state when the trigger pulled.

Why atomic snapshots aren’t atomic application states

A snapshot only catches the disk at a specific microsecond. If your database had half-written WAL logs or a pending transaction that got cut off mid-flush, that snapshot is technically valid at the block level but logically corrupt. Most sysadmins ignore the freeze phase of the backup process, assuming the kernel handles everything. It does not. If you aren’t signalling your application to flush its buffers and pause I/O, you’re rolling dice every time you run a backup job.

fsfreeze -f /var/lib/mysql && lvcreate -s -L 10G -n backup_snap /dev/vg0/data && fsfreeze -u /var/lib/mysql

• fsfreeze puts the underlying filesystem into a consistent state before the snapshot runs
• Always test the restored snapshot mount point before assuming the backup was successful
• Don’t rely on the snapshot being readable if the underlying host crashed mid-write

Verify or it never happened

If you don’t have an automated routine that spins up these snapshots in an isolated network to run a consistency check, your recovery plan is just a theory. Scripting a loop to mount these snapshots and run an fsck or a database-specific integrity check is tedious, but it saves your job when the primary drive inevitably reports read errors on a sector you haven’t touched in six months. If you can’t restore it, don’t pretend you have a backup.