syslog, rsyslog, syslog-ng Description and Usage in Linux
1. Introduction
syslog, rsyslog, and syslog-ng are different mechanism available to capture the system logs of different hosts and provides an ability to collect them on a central server.
SYSLOG stands for SYStem LOGs and is responsible to capture and collect system logs.
Syslog server is responsible to collect the logs from various syslog clients which send their system logs via UDP port 514.
It is widely used as it the simplest solution to achieve the task of collecting logs, syslog messages are simply sent to the syslog server regardless if there the server is configured on the other end or not.
2. History & Features
1980 –> Syslog
1998 –> Syslog-ng
2004 –> rsyslog
Syslog project was the first project developed as a part of sendmail implementation for collecting system logs and was very simple to use and configure.
It is configured to function only on UDP port 514, and since only UDP protocol is used, it doesn’t guarantee the data transferred.
syslog-ng i.e. Syslog New Generation was introduced later in 1990s, and extends the syslog functionality with new features like:
a. TCP Protocol support for transport
b. Content-based filtering
c. Database Logging feature
d. TLS encryption
Rsyslog was finally introduced in 2004. It extends the syslog protocol with few other features like:
a. Buffered operation support
b. Support for RELP Protocol
These three projects are growing up separately by different communities with different versions, and grown in parallel.
rsyslog is preferred nowadays with the most mature projects offering the features we usually need with simplicity and ease of use.
3. Syslog Message Format & Configuration
Syslog client configuration include configuring the /etc/syslog.conf, or, /etc/syslog-ng.conf, or /etc/rsyslog.conf file.
Lets see the file for example:-
a. Syslog Client Config File
[root@nglinux etc]# cat rsyslog.conf # rsyslog v5 configuration file # For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html # If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html #### MODULES #### $ModLoad imuxsock # provides support for local system logging (e.g. via logger command) $ModLoad imklog # provides kernel logging support (previously done by rklogd) #$ModLoad immark # provides --MARK-- message capability # Provides UDP syslog reception #$ModLoad imudp #$UDPServerRun 514 # Provides TCP syslog reception #$ModLoad imtcp #$InputTCPServerRun 514 #### GLOBAL DIRECTIVES #### # Use default timestamp format $ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat # File syncing capability is disabled by default. This feature is usually not required, # not useful and an extreme performance hit #$ActionFileEnableSync on # Include all config files in /etc/rsyslog.d/ $IncludeConfig /etc/rsyslog.d/*.conf #### RULES #### # Log all kernel messages to the console. # Logging much else clutters up the screen. #kern.* /dev/console # Log anything (except mail) of level info or higher. # Don't log private authentication messages! *.info;mail.none;authpriv.none;cron.none /var/log/messages # The authpriv file has restricted access. authpriv.* /var/log/secure # Log all the mail messages in one place. mail.* -/var/log/maillog # Log cron stuff cron.* /var/log/cron # Everybody gets emergency messages *.emerg * # Save news errors of level crit and higher in a special file. uucp,news.crit /var/log/spooler # Save boot messages also to boot.log local7.* /var/log/boot.log # ### begin forwarding rule ### # The statement between the begin ... end define a SINGLE forwarding # rule. They belong together, do NOT split them. If you create multiple # forwarding rules, duplicate the whole block! # Remote Logging (we use TCP for reliable delivery) # # An on-disk queue is created for this action. If the remote host is # down, messages are spooled to disk and sent when it is up again. #$WorkDirectory /var/lib/rsyslog # where to place spool files #$ActionQueueFileName fwdRule1 # unique name prefix for spool files #$ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible) #$ActionQueueSaveOnShutdown on # save messages to disk on shutdown #$ActionQueueType LinkedList # run asynchronously #$ActionResumeRetryCount -1 # infinite retries if host is down # remote host is: name/ip:port, e.g. 192.168.0.1:514, port optional #*.* @@remote-host:514 # ### end of the forwarding rule ### [root@nglinux etc]#
In the above configuration, we can see different log files configured for logging different types of messages.
b. RSyslog Logs collected
We can view the logs collected by rsyslog below.
# tail /var/log/cron Dec 6 07:01:01 localhost run-parts(/etc/cron.hourly)[4912]: starting 0anacron Dec 6 07:01:01 localhost run-parts(/etc/cron.hourly)[4921]: finished 0anacron Dec 6 07:10:01 localhost CROND[4924]: (root) CMD (/usr/lib/sa/sa1 1 1) Dec 6 07:20:01 localhost CROND[4929]: (root) CMD (/usr/lib/sa/sa1 1 1) Dec 6 07:30:01 localhost CROND[4934]: (root) CMD (/usr/lib/sa/sa1 1 1)
c. rsyslog service status check
### In case service is stopped, please restart the service. [root@nglinux etc]# service rsyslog status rsyslogd (pid 1477) is running...
d. To send all syslogs to a remote server.
*.* @172.168.1.24:514 [or] *.* @logserver.ngelinux.com:514
4. RSyslog Server Setup
a. Uncomment last lines which are in bold.
[root@nglinux etc]# cat rsyslog.conf | more # rsyslog v5 configuration file # For more information see /usr/share/doc/rsyslog-*/rsyslog_conf.html # If you experience problems, see http://www.rsyslog.com/doc/troubleshoot.html #### MODULES #### $ModLoad imuxsock # provides support for local system logging (e.g. via logger command) $ModLoad imklog # provides kernel logging support (previously done by rklogd) #$ModLoad immark # provides --MARK-- message capability # Provides UDP syslog reception #$ModLoad imudp #$UDPServerRun 514 # Provides TCP syslog reception #$ModLoad imtcp #$InputTCPServerRun 514
The above lines will enable reception of logs on UDP/TCP port 514 and loads different modules required.
b. Restart rsyslog service
[root@nglinux etc]# service rsyslog restart Shutting down system logger: [ OK ] Starting system logger: [ OK ] [root@nglinux etc]#
c. If you have firewall/iptables enabled, then we need to add a rule to open TCP/UDP port 514.
d. In case of selinux enabled system, we need to enable syslog udp port 514.
# semanage -a -t syslogd_port_t -p udp 514
Thats it !! Now we have both client and server in working state.