I have defined a TCP input in inputs.conf
with the following corresponding stanza in props.conf
(Splunk Enterprise 6.4):
[source::tcp:6067]
KV_MODE = json
LINE_BREAKER = ((^[^{][^\r]*\r\n)*)\{\"[^}]+\}
SHOULD_LINEMERGE = false
If I send the following text to that port:
Preamble lines
That I do not want
To appear in the event
The following line is intentionally blank
{"myfield": "some_value"}
(with \r\n
at the end of each line)
I get two events in Splunk:
{"myfield": "some_value"}
, with myfield
correctly presented as a field (so, KV_MODE = json
is working).LINE_BREAKER
to discard!According to the props.conf
documentation:
The contents of the first capturing group are discarded, and will not be present in any event.
Yes, the contents of the first capturing group are discarded from the event I want... but they are present in that unwanted (and unexpected) event.
Why do I get that unwanted event? How do I prevent it?
I'm deliberately using the descriptive term "preamble" here, because I have previously attempted to do the same thing (discard those "preamble" lines) using PREAMBLE_REGEX
instead of LINE_BREAKER
:
[source::tcp:6067]
KV_MODE = json
HEADER_FIELD_LINE_NUMBER = 1
PREAMBLE_REGEX = ^[^{].*
but I cannot get PREAMBLE_REGEX
to work, no matter what combination of regex and preamble test cases I use; at least, not for a TCP input. I wonder whether PREAMBLE_REGEX
only applies to, say, file inputs, not TCP (or other network) inputs. The props.conf
documentation hints at this with the word "files":
Some files contain preamble lines.
but if it's true, I'd prefer that the documentation was more explicit (and this makes me wonder about the implicit limitations of other settings).
Try adding BREAK_ONLY_BEFORE
to make your LINE_BREAKER
less aggressive.