Logging & Journald

Stop systemd-journald from deleting logs when you actually need to debug a crash

🧩 The Challenge

Dealing with a production issue where the root cause happened three hours ago only to realize the journal has already rotated is enough to make anyone scream. Most default distros set a retention policy that effectively nukes your forensic evidence right when the ticket hits your queue.

💡 The Fix

You need to lock down the storage policy and define specific size limits in the journald configuration file so the system stops acting like logs are optional. This forces the daemon to respect your storage constraints instead of just nuking files based on a whim.

sed -i 's/#Storage=auto/Storage=persistent/' /etc/systemd/journald.conf &&
sed -i 's/#SystemMaxUse=/SystemMaxUse=2G/' /etc/systemd/journald.conf &&
systemctl restart systemd-journald

⚙️ Why It Works

Flipping Storage to persistent forces journald to write logs to /var/log/journal instead of just the ephemeral /run directory, while SystemMaxUse ensures you don’t fill up the entire partition while trying to be helpful.

🚀 Pro-Tip: Always verify the new settings with journalctl –disk-usage after you restart the service.

Linux Tips & Tricks | © ngelinux.com | 8/1/2026