What are some tips for optimizing a really busy loghost running syslog-ng?
In no particular order: • If you use DNS, at least keep a caching DNS server running on the local host and make use of it – or better yet don’t use DNS. You can post-process logs on an analysis host later on and resolve hostnames at that time if you need to. On your loghost your main concern is keeping up with the incoming log stream – the last thing you want to do is make the recording of events rely on an external lookup. syslog-ng blocks on DNS lookups (as noted elsewhere in this FAQ), so you’ll slow down/stop ALL destinations with slow/failed DNS lookups. • Don’t log to the console or a tty, under heavy load they won’t be able to read the messages as fast as syslog-ng sends them, slowing down syslog-ng too much. • Don’t use regular expressions in your filters. Instead of: filter f_xntp_filter_no_regexp { # original line: “xntpd[1567]: time error -1159.777379 is way too large (set clock manually); program(“xntpd”) and match(“time error .* is way too large .* set clock manually”); };
