Logging & Journald

Stop journald from losing your logs during a hard crash

🧩 The Challenge

Everyone has dealt with a server that panicked at 3 AM, and when you rebooted, the logs from the crash were just gone. You go to look for the tail end of the kernel panic, and journald is suddenly empty because it was only keeping the logs in a volatile memory buffer.

💡 The Fix

You need to force the journal to write to the persistent storage directory on disk instead of relying on the default volatile /run/log/journal location. It’s a simple directory creation that tells the system to stop keeping everything in RAM.

mkdir -p /var/log/journal && systemd-tmpfiles --create --prefix /var/log/journal && systemctl restart systemd-journald

⚙️ Why It Works

Creating that directory forces the daemon to switch from transient memory-only storage to the persistent filesystem. Systemd checks for the existence of that path during startup and automatically dumps the stream there if it’s writable.

🚀 Pro-Tip: Set Storage=persistent in /etc/systemd/journald.conf after you do this to make sure it sticks even after a package update.

Linux Tips & Tricks | © ngelinux.com | 8/2/2026