Rsyslog

From WhyAskWhy.org Wiki
Revision as of 13:50, 16 February 2015 by Deoren (talk | contribs) (Spacing tweak)
Jump to: navigation, search


Overview

From the rsyslog homepage:

Rsyslog is a rocket-fast system for log processing. It offers high-performance, great security features and a modular design. While it started as a regular syslogd, rsyslog has evolved into a kind of swiss army knife of logging, being able to

accept inputs from a wide variety of sources, transform them, and output the results to diverse destinations.

Rsyslog has a strong enterprise focus but also scales down to small systems. It supports, among others, MySQL, PostgreSQL, failover log destinations, syslog/tcp transport, fine grain output format control, high precision timestamps, queued operations and the ability to filter on any message part.


Filter Conditions

Rsyslog supports three different types of filter conditions [1]:

  • RainerScript-based filters
  • "traditional" severity and facility based selectors
  • property-based filters


RainerScript-based filters

if $programname == 'prog1' then {
   action(type="omfile" file="/var/log/prog1.log")
   if $msg contains 'test' then
     action(type="omfile" file="/var/log/prog1test.log")
   else
     action(type="omfile" file="/var/log/prog1notest.log")
}


Selectors

#
# Split it up so that it is easy to write scripts to parse these files.
#
#mail.info                      -/var/log/mail.info
#mail.warn                      -/var/log/mail.warn
mail.err                        /var/log/mail.err

# Catch-all mail log
mail.*                          -/var/log/mail.log

# Prevent mail log entries from being duplicated to /var/log/syslog
& stop


Property-based filters

[2]

:syslogtag, isequal, "mysqld:" -/var/log/mysqld.log
& ~
:syslogtag, isequal, "mysqld_safe:" -/var/log/mysqld.log
& ~
:syslogtag, startswith, "/etc/mysql/debian-start" -/var/log/mysqld.log
& ~


Discarding messages after they are logged (aka, stopping logging of repeat/duplicate messages)

The discard operator

The discard operator (aka, "discard action" operator) is the ~ character. Using it tells rsyslog that you want to discard log messages matched by filters. That operator is used in conjunction with the & character which allows for having multiple actions per selector. This allows for combinations where you direct matching log messages to a specific file and prevent the same message from being logged a second time. The & character is necessary (I found out the hard way) to "glue" the discard action to the filter above, otherwise it stops all log messages from being logged from that point forward. This is particularly important with

The stop directive

As of v7-stable the discard operator has been marked as deprecated and a warning similar to this one is logged:

 rsyslogd-2307: warning: ~ action is deprecated, consider using the 'stop' statement instead [try http://www.rsyslog.com/e/2307 ]

From v7 onward, the stop RainerScript directive is recommended instead. From my testing I have found that it works well with RainerScript, Selector or Property-based filters. If using it with RainerScript it doesn't appear that you need to use & to tie the directive to the previous filter like you do with selector or property-based filters. As with the discard operator (~), with those two filter approaches you run the risk of stopping all logging after the use of the directive if you don't tie it back to a preceding filter.

Example: Using the discard operator

From an Ubuntu 12.04 LTS box that runs v5.x. Here we're using a property-based filter.

# /etc/rsyslog.d/20-ufw.conf

# Log kernel generated UFW log messages to file
:msg,contains,"[UFW " /var/log/ufw.log

# Uncomment the following to stop logging anything that matches the last rule.
# Doing this will stop logging kernel generated UFW log messages to the file
# normally containing kern.* messages (eg, /var/log/kern.log)
& ~


Example: Using the stop operator

From an Ubuntu 14.04 LTS box that runs v7.4.x. Here we're also using a property-based filter.

# /etc/rsyslog.d/20-ufw.conf

:msg,contains,"[UFW " /var/log/ufw.log

# Uncomment the following to stop logging anything that matches the last rule.
# Doing this will stop logging kernel generated UFW log messages to the file
# normally containing kern.* messages (eg, /var/log/kern.log)
& stop


References

General

Rsyslog actions

Filters

Regular Expressions

Properties

Receiving from and Sending to remote Syslog boxes