How do we assign each JSON document to a distinct event?

[ddrillic]

We have a case in which multiple json documents are being clamped together into one Splunk event. How do we untangle it?

[somesoni2]

You would need to set appropriate Line breaking configuration for your sourcetype, and for which we'd need some sample data (mask anything that's sensitive), and some details on how you'd want to break that sample event.

[ddrillic]

It looks like -

{"userDetails":{sensitive data},"message":null}
{"userDetails":{sensitive data},"message":null}
{"userDetails":{sensitive data},"message":null}
{"userDetails":{sensitive data},"message":null}

[somesoni2]

Try to use following in props.conf on Indexer(s)/Heavy Forwarder(s) whichever comes first.

[YourSourceTypeHere]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)(?=\{\"userDetails\"\:)
..other timestamp extraction attributes...

[ddrillic]

Gorgeous as usual ; -)
But, any way to avoid the hard-coding of userDetails?

Needless to say - working as expected !!!!!!!!!!

[somesoni2]

Well, you generally need to put an anchor for identifying line start. You can try with ([\r\n]+)(?=\{\"\w+\"\:) to see if that works for. Since we don't have full events, we can't say for sure that it'll work (there may be other entries matching that pattern).

[DavidHourani]

Hi ddrillic,

This usually happens when you have brackets at the beginning of your JSON containing the entire document. It makes it as if the entire document is a value for one of the elements. You should set up a sedcmd in your props to clear this up, or clear it via script before the data gets into Splunk.

If you post a copy of the header/end of your JSON file I can help you set up the sedcmd.

Regards,
David

[ddrillic]

Interesting - it looks like {"userDetails":{...."message":null} followed by another one like this one - {"userDetails":{...."message":null}...

[DavidHourani]

if your lines are always starting with a new element you can go for this config :

[yourSourcetype]
BREAK_ONLY_BEFORE = ^\{

[skoelpin]

LINE_BREAKER would be a much better approach than BREAK_ONLY_BEFORE

[DavidHourani]

why do you say that ?

[skoelpin]

If you set SHOULD_LINEMERGE = false and use LINE_BREAKER, this will skip the merging pipeline and give a performance boost

http://wiki.splunk.com/Community:HowIndexingWorks

[harsmarvania57]

Hi @ddrillic,

Can you please provide some sample data?

[niketn]

@ddrillic also add what is your current sourcetype stanza for JSON data?

[ddrillic]

@niketnilay, sorry for the delay. We didn't set anything in the configuration files.