We are logging the following application network statistics. I want to be able to index the data into splunk so we can generate reports on it.
The First line consists of the following fields:
timestamp, site name, remote server name , local server name
Other lines of the same record consists of the following fields:
statistic name : message type : origin Node : statistic Value
This is the actual log:
1386704158913 SITE-A,remoteServer1,localhost
receivedMessages:AAA:NODE1:10
receivedMessages:BBB:NODE1:10
sentMessages:CCC:NODE2:10
discMessages:AAA:NODE1:1
discMessages:BBB:NODE2:1
1386704158913 SITE-A,remoteServer2,localhost2
receivedMessages:FFF:NODE1:10
receivedMessages:GGG:NODE1:10
sentMessages:HHH:NODE2:10
discMessages:FFF:NODE1:1
discMessages:III:NODE2:1
Is there a way to extract all the fields above from that log format?
Thanks a lot.
Use Regext to extract the time, then MVEXPAND then you will be able to have the correlation. Then make the extractions.
You can easily do that by adjusting the line breaking in props.conf
Have a play with regular expressions and the options under "Attributes that are available only when SHOULD_LINEMERGE is set to true" in
http://docs.splunk.com/Documentation/Splunk/6.0/Data/Indexmulti-lineevents
No, you should not break them into single-line events, for the exact reasons that you mention. My question was if you had succeeded in creating the (multi-line) events correctly in splunk.
If i break the log/lines into individual events, wont i loos the correlation between the first line (which consists of the event time stamp and other shared fields) and the other sub lines?
Any problems with breaking the log into indvidual events? Or is it only regarding the field extraction?