Most of the time we use a shared report ("General Product Report") to view our logs for sourcetype="product". I created a field extraction rule to parse each entry into 7-8 fields (the sample below has been trimmed down for brevity).
^(?P<ts>.{23})\s(?P<level>[A-Z]{4,6})\s+\[\s*(?P<tid>.+)\]\s+:->\s(?P<body>.*)$
I'd like to create another shared report "Product Performance Report" that parses the same sourcetype differently as roughly 30% of the entries in product log contain performance data that we would like to chart. This extraction pulls out the 'duration' and 'url' fields from those entries.
^(?P<ts>.{23})\s(?P<level>[A-Z]{4,6})\s+\[\s*(?P<tid>.+)\]\s+:->\s\-\-Done\s\[(?P<dur>.*)\]\s\[(?P<url>.*)\].*$
How can I apply the 2nd extraction 'rule' to the same sourcetype but only use it when viewing the "Performance Report"? Is there a better approach to get the same results?
Sample Entries:
2015-01-23 00:02:06,161 INFO [ 68] 😆 foo bar
2015-01-23 00:02:26,177 INFO [ 65] 😆 --Done [ 15.581] [http://the.url.org/mickey/mouse]
2015-01-23 00:02:36,302 INFO [ 65] 😆 bla bla bla
2015-01-23 00:02:36,349 INFO [ 65] 😆 --Done [ 203.111] [http://the.url.org/donald/duck]
The field extraction is done at sourcetype level so I am not sure if you can conditionally choose which field extractions to use. What I would suggest is the define two field extractiosn stanza, for your sourcetype.
Props.conf in Search Head
[product]
EXTRACT-general = ^(?P<ts>.{23})\s(?P<level>[A-Z]{4,6})\s+\[\s*(?P<tid>.+)\]\s+:->\s(?P<body>.*)$
EXTRACT-perf = :->\s\-\-Done\s\[(?P<dur>.*)\]\s\[(?P<url>.*)\].*$
This should create fields ts, level,tid, body for all events. It will also create dur and url for all events but for non performance data, they would be null. So, for General report your just refer fields ts, level,tid, body and for Performance report, just use fields ts, level,tid, dur, url.
The field extraction is done at sourcetype level so I am not sure if you can conditionally choose which field extractions to use. What I would suggest is the define two field extractiosn stanza, for your sourcetype.
Props.conf in Search Head
[product]
EXTRACT-general = ^(?P<ts>.{23})\s(?P<level>[A-Z]{4,6})\s+\[\s*(?P<tid>.+)\]\s+:->\s(?P<body>.*)$
EXTRACT-perf = :->\s\-\-Done\s\[(?P<dur>.*)\]\s\[(?P<url>.*)\].*$
This should create fields ts, level,tid, body for all events. It will also create dur and url for all events but for non performance data, they would be null. So, for General report your just refer fields ts, level,tid, body and for Performance report, just use fields ts, level,tid, dur, url.
Thanks. That makes some sense. I'll give that a try. Can I do that through the admin UI? My operation's staff doesn't give me direct access to props.conf.
Yes, You can add field extraction through Splunk Web's admin pages.
http://docs.splunk.com/Documentation/Splunk/6.2.1/Knowledge/Managesearch-timefieldextractions
Make your comment an 'answer' so I can mark the question as answered. Thanks.
Glad it helped. Here you go.
It worked. Thanks.