I have an ongoing problem that I hope just goes away when I upgrade completely to v4. My current setup is v3 Forwarders sending data to v3 indexer, which is storing and forwarding all results to a v4 indexer.
Every once in a while, logs end up indexed with multiple events crammed together, ignoring the BREAK_ONLY_BEFORE pattern. I of course cannot reproduce the problem outside of production. I'm sure it is something to do with overflowing some buffer somewhere.
My props.conf looks like this:
<code>[rm3] BREAK_ONLY_BEFORE=20[0-9][0-9]-d+-d+s+d+:d+:d+,d+s+ pulldown_type = true AUTO_TAG = false KV_MODE = none MAX_TIMESTAMP_LOOKAHEAD = 25 MAX_EVENTS = 512 AUTO_LINEMERGE = false </code>
Anyway, I'm hoping that in the short term, there is some command that can split up results based on some pattern at search time. In this case, I want to break on ^2010. If this doesn't exist, I'll make a command for it, I was just hoping something already exists.
There's no good way to do this at search time since field extraction is run before you'd have a chance to do anything meaningful to the events. A technique like this can be used to split separate lines into separate results, but it's filled with problems:
<code>... | rex mode=sed "s/n/NL/g" | eval raw=_raw | makemv raw delim="NL" | mvexpand raw | eval _raw = raw </code>
Some of the big problems are: