I have some suricata stats logs which are in the following format:
------------------------------------------------------------------
Date: 9/26/2012 -- 16:57:53 (uptime: 0d, 00h 00m 24s)
------------------------------------------------------------------
Counter | TM Name | Value
------------------------------------------------------------------
tcp.sessions | Detect | 2932
tcp.blah | Whatever | 42
...
string.string | string | 0
------------------------------------------------------------------
I'm trying to :
...but I'm not getting it done
My props.conf looks like:
[suricata_stats]
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = true
BREAK_ONLY_BEFORE_DATE = false
BREAK_ONLY_BEFORE = \-*\r\nDate
EXTRACT-counter = ^(?<counter>[a-z]+\.[a-z]+)\s+|
EXTRACT-module = ^[a-z]+\.[a-z]+\s+|\s+(?<module>[^ ])\s+|
EXTRACT-value = ^[a-z]+\.[a-z]+\s+|\s+[^ ])\s+|\s+(?P<value>.+)
I know the regex is .. messy, but the results I'm getting are not as intended.
With sample data containing what i intend to be 5 results, I get 6.
One event is :
--------------------------------------------------------------------------
The next five events begin with "Date" and display all of the subsequent lines.
My extractions are also wrong:
Where do we start?
I'm working on the same thing, actually. I'm curious if you've managed to get a good setup for this. Here's what I have in my props.conf:
[suricata_stats_log2]
BREAK_ONLY_BEFORE = Date:
NO_BINARY_CHECK = 1
SHOULD_LINEMERGE = False
TIME_PREFIX = Date:
pulldown_type = 1
EXTRACT-Counter = (?i)(?P<Counter>[^ ]+)\s+\|\s+\w+\s+\|\s+\d+
EXTRACT-TM_Name = (?i)\..*? \| (?P<TM_Name>\w+)(?= )
EXTRACT-Value = (?i) .*? \| (?P<Value>\d+)
Each line (tcp.sessions | Detect | 2932, etc) is it's own event with the correct timestamp. However, I'm not sure I like this particular solution yet. 272 actual stats turn into over 27000 events in Splunk! I'm still futzing with it to try to find something I like.
I've tried multikv and either it didn't work at all, or I was totally broken in my search implementation of it, heh.
There is the multikv command to parse out tabular data at search-time, which would likely be easier than trying to do that in props.conf. But there is still the issue of getting the event in correctly in the first place, which I'm not sure about...