That LVM Snapshot Backup? Have You Tried Recovering From It?
By Saket Jain Published Linux/Unix
That LVM Snapshot Backup? Have You Tried Recovering From It?
Technical Briefing | 9/24/2026
We’ve all been there: you set up your LVM snapshots, configure a daily dd or rsync of the snapshot logical volume, and tick the backup box. Feels good, right? Like a safety net. But I’ve got to ask: when was the last time you actually *restored* that full system from scratch? Not just mounted the filesystem and checked for files, but genuinely tried to boot it on new hardware or a fresh VM?
The Silent Killer: Identical Filesystem UUIDs
See, when you dd a block device, you’re getting an exact byte-for-byte copy. That includes the filesystem’s unique identifier, its UUID. Every filesystem gets one, and the kernel uses it to figure out which partition is which, especially for / and other critical mount points defined in /etc/fstab. If you try to bring up a system with two identical root filesystem UUIDs, even if one is on the old disk and the other on a new one, the kernel gets confused. It’s like having two kids named ‘Billy’ in the same room and shouting for one of them; chaos ensues. You’ll likely end up in an initramfs emergency shell, scratching your head, wondering why your carefully restored system won’t boot.
This bit me in prod during a particularly nasty disk failure a few years back. The dd backup was there, complete, but the restored system just wouldn’t come up cleanly. We spent hours debugging what looked like fstab errors before realizing the kernel was just seeing duplicate UUIDs. Most tutorials focus on creating the snapshot, but they rarely walk you through the nitty-gritty of making that restored block device *actually bootable* and functional in a new environment.
sudo tune2fs -U random /dev/vg/restored_root_lv
sudo blkid /dev/vg/restored_root_lv
- Restore the backup to a new block device (e.g., a new LVM logical volume or disk partition).
- Change the restored filesystem’s UUID to a fresh, random one using the `tune2fs` command shown above.
- Mount the restored filesystem (e.g., `sudo mount /dev/your/restored_volume /mnt/recovery`), then edit its `/etc/fstab` to reference the *new* UUID you got from `blkid`.
- If it’s a bootable system, `chroot` into your recovery mount point, update GRUB configuration, and regenerate your `initramfs` (e.g., `update-grub`, `update-initramfs -u`).
- Finally, test boot the recovered system. Seriously, don’t just assume it works.
Beyond LVM: The Principle Holds
And look, this isn’t just an LVM-specific quirk. You might use fsarchiver, partclone, or even just a hefty `rsync -ax` of /. The moment you restore a full OS filesystem to new hardware, or even a new LVM logical volume, you’re going to hit similar issues with how the system identifies and mounts its root filesystem. Permissions, device files, bootloader setup — they all demand attention beyond just copying bytes.
The biggest takeaway here? Don’t just back it up, *practice restoring it*. And not just `tar -tf` to check filenames, but a full system restore to a different piece of hardware or a VM. It’ll expose these subtle configuration gotchas long before you’re in the middle of a screaming production outage. You’ll thank yourself for the foresight.
