Zimbra monitoring with Splunk

[orbiterone]

I've got Splunk installed on a Linux system and I'm forwarding all of the logs from my Zimbra email server over to splunk using Splunk to listen on UDP 514. The logs are being captured as syslog events and tagged with the host name.

The Zimbra logs are actually being written as CSV events similar to the below:

[host data and timestamp info excluded] zimbramon[3207]: 3207:info: zmstat mtaqueue.csv: timestamp, KBytes, requests:: 10/04/2011 10:27:38, 0, 0

[host data and timestamp info excluded] zimbramon[3191]: 3191:info: zmstat cpu.csv: timestamp, cpu:user, cpu:nice, cpu:sys, cpu:idle, cpu:iowait, cpu:irq, cpu:softirq, cpu0:user, cpu0:nice, cpu0:sys, cpu0:idle, cpu0:iowait, cpu0:irq, cpu0:softirq:: 10/04/2011 10:27:30, 4.2, 0.0, 1.8, 93.6, 0.4, 0.0, 0.0, 4.2, 0.0, 1.8, 93.6, 0.4, 0.0, 0.0

Each type of output has it's own csv format, with the header included in the log event.

Any tips on how to categorize each of these for more efficient field tagging, and possibly even charting? As you can see for performance monitoring it can track the mail queue size and also provides cpu stats. Once I get them categorized I can also look at creating alerts when an event gets to high or stays to high.

[kurta]

These are not Zimbra logs. You are looking at the zmstats files. They definitely contain useful information but the logs are usually under /opt/zimbra/log and named something like mailbox.log. There is also an access log tracking the web server connections (location varies by Zimbra version and you can also adjust via configuration). If you are running http or pop proxies, you will also have nginx log files.

[OL]

Hello,

The way I would do is using a rex command to extract the values after the date at search time such as:

index=-the index you are sending to- source=-your source file- "zmstat cpu.csv" | rex "softirq:: [^,]+,(?[0-9.]),(?cpu:nice[0-9.]),(?cpu:sys[0-9.]),(?cpu:idle[0-9.]) -and so on-"

If you are a bit advanced, you could as well ignore the header at index time to save some license space if this is important for you!

Hope it helps.
Olivier