Stop ZFS from eating your entire pool when you enable dedup

Technical Briefing | 8/4/2026

You see a massive pile of identical virtual machine images or container layers and think, deduplication is free storage. It looks like a win on paper. But then you enable it, and suddenly your pool’s latency spikes, your RAM usage goes into orbit, and writes that used to take milliseconds start feeling like they’re crawling through molasses. That’s because deduplication in ZFS happens at record time, and the deduplication table must reside in your ARC. If it doesn’t fit, you’re hitting the disk for every single block write.

When your RAM is not enough

The ZFS DDT is a giant hash map. If you enable deduplication, every time you write a block, ZFS calculates a checksum and looks it up in the DDT. If that DDT isn’t in memory, it’s a synchronous disk read before the write can proceed. Most people think they have plenty of RAM until the DDT grows beyond the size of their physical DIMMs. Once that happens, performance craters hard. I have seen production systems ground to a halt because the storage subsystem was waiting on random I/O for the metadata lookups.

zpool status -D yourpoolname

• The -D flag shows you the actual DDT size
• Watch the ratio column to see if you are actually saving space
• Check your hit rate to see if the table is staying cached
• Only enable this on pools with massive over-provisioned RAM

Why you should just use compression instead

Lz4 is your best friend here. It is fast, efficient, and handles repetitive data streams better than most people give it credit for. It doesn’t come with the crippling metadata overhead of deduplication. Unless you have a very specific, read-heavy workload with an insane amount of duplicate blocks that you absolutely must minimize on disk, stick to compression. If you already turned dedup on, turning it off won’t retroactively fix your old data. You have to send the data to a new dataset or pool without the property enabled to actually reclaim that performance.

Next time you feel the urge to optimize your disk usage, start with a zfs get all and check your compression ratios. If that isn’t cutting it, you probably have a bigger issue with your data layout. Don’t touch the dedup switch unless you have a staging environment that can mirror your production load, or you will eventually be the one explaining to management why the storage is suddenly slower than an old spinning floppy drive.