I've run back and forth through the props.conf documentation and done a few circuits of Answers, but I haven't found anything that actually works yet, so here we are.
I have these large multi-line events coming into splunk with source=tcp-raw. They look kind of like this:
@@@
-> section 1
*bunch of stuff here*
@@@
-> section 2
*bunch of stuff here*
@@@
and so on. The "@@@" is what I want to break the events up on, since it's an obvious delimiter. So I tried adding the following to props.conf
[source::tcp-raw]
SHOULD_LINEMERGE = false
DATETIME_CONFIG = current
LINE_BREAKER = @@@
The result was splunk either not indexing the data at all, or throwing it out somehow. Whenever I'd send a new report with that stanza in place, nothing would show up. Removed it, and the data appeared in its normal unbroken form.
What am I doing wrong?
I've solved my own problem. The stanza now looks like:
BREAK_ONLY_BEFORE = @@@
SHOULD_LINEMERGE = true
It's now splitting properly.
Since I'm doing this in a test environment I'm able to throw a new batch of data at it on demand, which should then be indexed under the new rules. I'm following this process: change props.conf in /local, restart splunk server, send new data, check result, repeat.
Did you re-index all the data or are you looking at the same data? It needs to be re-indexed.
OK, that fixed the problem of splunk throwing out the incoming data, but it's still not breaking up the events.
Try "BREAK_ONLY_BEFORE = @@@" instead of "LINE_BREAKER = @@@"