RHEL 7: How to use journalctl to examine system logs and its various options ?

Today we will look how to use journalctl command to examine system logs in RHEL 7 or 8.

And various options which are used frequently.

1. Journalctl command displays all logs in paginated view.

[root@ngelinux01 ~]# journalctl
-- Logs begin at Fri 2021-06-11 07:25:25 BST, end at Fri 2021-06-11 07:27:06 BST. --
Jun 11 07:25:25 host-0-0.linuxzoo.net systemd-journal[86]: Runtime journal is using 4.0M (max allowed 24.4M, trying to leave 36.6M free of 240.
Jun 11 07:25:25 host-0-0.linuxzoo.net kernel: Initializing cgroup subsys cpuset
Jun 11 07:25:25 host-0-0.linuxzoo.net kernel: Initializing cgroup subsys cpu
Jun 11 07:25:25 host-0-0.linuxzoo.net kernel: Initializing cgroup subsys cpuacct
Jun 11 07:25:25 host-0-0.linuxzoo.net kernel: Linux version 3.10.0-514.10.2.el7.x86_64 (builder@kbuilder.dev.centos.org) (gcc version 4.8.5 201
Jun 11 07:25:25 host-0-0.linuxzoo.net kernel: Command line: BOOT_IMAGE=/vmlinuz-3.10.0-514.10.2.el7.x86_64 root=/dev/mapper/centos_lvm-root ro
Jun 11 07:25:25 host-0-0.linuxzoo.net kernel: e820: BIOS-provided physical RAM map:
Jun 11 07:25:25 host-0-0.linuxzoo.net kernel: BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
Jun 11 07:25:25 host-0-0.linuxzoo.net kernel: BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
Jun 11 07:25:25 host-0-0.linuxzoo.net kernel: BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved

 

2. To see new logs first, we can reverse the output.

[root@ngelinux01 ~]# journalctl
-- Logs begin at Fri 2021-06-11 07:25:25 BST, end at Fri 2021-06-11 07:27:06 BST. --
Jun 11 07:25:25 host-0-0.linuxzoo.net systemd-journal[86]: Runtime journal is using 4.0M (max allowed 24.4M, trying to leave 36.6M free of 240.
Jun 11 07:25:25 host-0-0.linuxzoo.net kernel: Initializing cgroup subsys cpuset
Jun 11 07:25:25 host-0-0.linuxzoo.net kernel: Initializing cgroup subsys cpu
Jun 11 07:25:25 host-0-0.linuxzoo.net kernel: Initializing cgroup subsys cpuacct
Jun 11 07:25:25 host-0-0.linuxzoo.net kernel: Linux version 3.10.0-514.10.2.el7.x86_64 (builder@kbuilder.dev.centos.org) (gcc version 4.8.5 201
Jun 11 07:25:25 host-0-0.linuxzoo.net kernel: Command line: BOOT_IMAGE=/vmlinuz-3.10.0-514.10.2.el7.x86_64 root=/dev/mapper/centos_lvm-root ro
Jun 11 07:25:25 host-0-0.linuxzoo.net kernel: e820: BIOS-provided physical RAM map:
Jun 11 07:25:25 host-0-0.linuxzoo.net kernel: BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
Jun 11 07:25:25 host-0-0.linuxzoo.net kernel: BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
Jun 11 07:25:25 host-0-0.linuxzoo.net kernel: BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved

 

3. To display latest 8 lines of log only.

[root@ngelinux01 ~]# journalctl -r -n 8
-- Logs begin at Fri 2021-06-11 07:25:25 BST, end at Fri 2021-06-11 07:27:06 BST. --
Jun 11 07:27:06 ngelinux01.linuxzoo.net systemd[1]: Started Session 1 of user root.
Jun 11 07:27:06 ngelinux01.linuxzoo.net systemd[1]: Starting user-0.slice.
Jun 11 07:27:06 ngelinux01.linuxzoo.net systemd[1]: Created slice user-0.slice.
Jun 11 07:27:06 ngelinux01.linuxzoo.net sshd[1661]: Accepted password for root from 10.200.0.1 port 55248 ssh2
Jun 11 07:26:42 ngelinux01.linuxzoo.net realmd[1600]: stopping service
Jun 11 07:26:42 ngelinux01.linuxzoo.net realmd[1600]: quitting realmd service after timeout
Jun 11 07:26:40 ngelinux01.linuxzoo.net sshd[1661]: dispatch_protocol_error: type 98 seq 5 [preauth]
Jun 11 07:26:11 ngelinux01.linuxzoo.net fprintd[1598]: ** Message: No devices in use, exit
[root@ngelinux01 ~]#

 

4. To get specific log of warning type.

root@ngelinux01 ~]# journalctl -p warning
-- Logs begin at Fri 2021-06-11 07:25:25 BST, end at Fri 2021-06-11 07:48:06 BST. --
Jun 11 07:25:25 host-0-0.linuxzoo.net kernel: ACPI: RSDP 00000000000f69c0 00014 (v00 BOCHS )
Jun 11 07:25:25 host-0-0.linuxzoo.net kernel: ACPI: RSDT 000000001ffe18dc 00030 (v01 BOCHS  BXPCRSDT 00000001 BXPC 00000001)
Jun 11 07:25:25 host-0-0.linuxzoo.net kernel: ACPI: FACP 000000001ffe17b8 00074 (v01 BOCHS  BXPCFACP 00000001 BXPC 00000001)
Jun 11 07:25:25 host-0-0.linuxzoo.net kernel: ACPI: DSDT 000000001ffe0040 01778 (v01 BOCHS  BXPCDSDT 00000001 BXPC 00000001)
Jun 11 07:25:25 host-0-0.linuxzoo.net kernel: ACPI: FACS 000000001ffe0000 00040
Jun 11 07:25:25 host-0-0.linuxzoo.net kernel: ACPI: APIC 000000001ffe182c 00078 (v01 BOCHS  BXPCAPIC 00000001 BXPC 00000001)
Jun 11 07:25:25 host-0-0.linuxzoo.net kernel: ACPI: HPET 000000001ffe18a4 00038 (v01 BOCHS  BXPCHPET 00000001 BXPC 00000001)
Jun 11 07:25:25 host-0-0.linuxzoo.net kernel: kexec: crashkernel=auto resulted in zero bytes of reserved memory.
Jun 11 07:25:25 host-0-0.linuxzoo.net kernel: Zone ranges:
Jun 11 07:25:25 host-0-0.linuxzoo.net kernel:   DMA      [mem 0x00001000-0x00ffffff]

 

5. To see all options of journalctl command.

[root@ngelinux01 ~]# journalctl --help
journalctl [OPTIONS...] [MATCHES...]

Query the journal.

Flags:
     --system              Show the system journal
     --user                Show the user journal for the current user
  -M --machine=CONTAINER   Operate on local container
  -S --since=DATE          Show entries not older than the specified date
  -U --until=DATE          Show entries not newer than the specified date
  -c --cursor=CURSOR       Show entries starting at the specified cursor
     --after-cursor=CURSOR Show entries after the specified cursor

 

6. Journal Logs saves only current boot messages.
To save the messages from past boots also, we can make following changes.

#### First step is to create below directory
# mkdir -p /var/log/journal

#### Make "storage=persistent" from "storage=auto" like below.
[root@ngelinux01 ~]# more /etc/systemd/journald.conf
#  This file is part of systemd.
#
#  systemd is free software; you can redistribute it and/or modify it
#  under the terms of the GNU Lesser General Public License as published by
#  the Free Software Foundation; either version 2.1 of the License, or
#  (at your option) any later version.
#
# Entries in this file show the compile time defaults.
# You can change settings by editing this file.
# Defaults can be restored by simply deleting this file.
#
# See journald.conf(5) for details.

[Journal]
Storage=persistent
#Compress=yes

##### Now restart the journalctl service
[root@ngelinux01 ~]# systemctl restart  systemd-journald.service
[root@ngelinux01 ~]#

 

7. View logs between specific time frames.

[root@ngelinux01 ~]# journalctl --since "2021-01-10" --until "2021-01-10 01:00"
-- Logs begin at Fri 2021-06-11 07:25:25 BST, end at Fri 2021-06-11 08:20:02 BST. --
[root@ngelinux01 ~]#

[root@ngelinux01 ~]# journalctl --since "2021-06-11 07:45" --until "2021-06-11 08:00:00"
-- Logs begin at Fri 2021-06-11 07:25:25 BST, end at Fri 2021-06-11 08:20:02 BST. --
Jun 11 07:46:40 ngelinux01.linuxzoo.net sshd[1892]: Bad packet length 2790760703. [preauth]
Jun 11 07:46:40 ngelinux01.linuxzoo.net sshd[1892]: Disconnecting: Packet corrupt [preauth]
Jun 11 07:46:58 ngelinux01.linuxzoo.net sshd[1894]: dispatch_protocol_error: type 98 seq 5 [preauth]
Jun 11 07:47:33 ngelinux01.linuxzoo.net sshd[1894]: error: buffer_get_ret: trying to get more bytes 4 than in buffer 0 [preauth]
Jun 11 07:47:33 ngelinux01.linuxzoo.net sshd[1894]: error: buffer_get_string_ret: cannot extract length [preauth]
Jun 11 07:47:33 ngelinux01.linuxzoo.net sshd[1894]: fatal: buffer_get_string: buffer error [preauth]
Jun 11 07:47:39 ngelinux01.linuxzoo.net sshd[1904]: dispatch_protocol_error: type 98 seq 5 [preauth]
Jun 11 07:47:45 ngelinux01.linuxzoo.net sshd[1904]: Accepted password for root from 10.200.0.1 port 56718 ssh2
Jun 11 07:47:45 ngelinux01.linuxzoo.net systemd[1]: Started Session 2 of user root.
Jun 11 07:47:45 ngelinux01.linuxzoo.net sshd[1904]: pam_unix(sshd:session): session opened for user root by (uid=0)
Jun 11 07:47:45 ngelinux01.linuxzoo.net systemd-logind[604]: New session 2 of user root.
Jun 11 07:47:45 ngelinux01.linuxzoo.net systemd[1]: Starting Session 2 of user root.
Jun 11 07:47:46 ngelinux01.linuxzoo.net dbus[619]: [system] Activating service name='org.freedesktop.problems' (using servicehelper)
Jun 11 07:47:46 ngelinux01.linuxzoo.net dbus-daemon[619]: dbus[619]: [system] Activating service name='org.freedesktop.problems' (using service
Jun 11 07:47:46 ngelinux01.linuxzoo.net dbus[619]: [system] Successfully activated service 'org.freedesktop.problems'
Jun 11 07:47:46 ngelinux01.linuxzoo.net dbus-daemon[619]: dbus[619]: [system] Successfully activated service 'org.freedesktop.problems'
Jun 11 07:48:06 ngelinux01.linuxzoo.net sshd[1661]: pam_unix(sshd:session): session closed for user root
Jun 11 07:48:06 ngelinux01.linuxzoo.net systemd-logind[604]: Removed session 1.
lines 1-19/19 (END)

8. To see logs of a specific unit of specific time.

[root@ngelinux01 ~]# journalctl -u cups.service --since "2021-06-11 12:30"
-- No entries --

 

9. To check logs of a specific PID.

[root@ngelinux01 ~]# journalctl _PID=1
-- Logs begin at Fri 2021-06-11 07:25:25 BST, end at Fri 2021-06-11 08:20:02 BST. --
Jun 11 07:25:25 host-0-0.linuxzoo.net systemd[1]: Started dracut cmdline hook.
Jun 11 07:25:25 host-0-0.linuxzoo.net systemd[1]: Starting dracut pre-udev hook...
Jun 11 07:25:25 host-0-0.linuxzoo.net systemd[1]: Started dracut pre-udev hook.
Jun 11 07:25:25 host-0-0.linuxzoo.net systemd[1]: Starting udev Kernel Device Manager...
Jun 11 07:25:25 host-0-0.linuxzoo.net systemd[1]: Started udev Kernel Device Manager.

Similarly we have various options, however the options mentioned above are most widely used and help us to serve the day to day purpose.

Leave a Reply

Your email address will not be published.