Getting Data In

How do we assign each JSON document to a distinct event?

ddrillic
Ultra Champion

We have a case in which multiple json documents are being clamped together into one Splunk event. How do we untangle it?

0 Karma

somesoni2
SplunkTrust
SplunkTrust

You would need to set appropriate Line breaking configuration for your sourcetype, and for which we'd need some sample data (mask anything that's sensitive), and some details on how you'd want to break that sample event.

0 Karma

ddrillic
Ultra Champion

It looks like -

{"userDetails":{sensitive data},"message":null}
{"userDetails":{sensitive data},"message":null}
{"userDetails":{sensitive data},"message":null}
{"userDetails":{sensitive data},"message":null}
0 Karma

somesoni2
SplunkTrust
SplunkTrust

Try to use following in props.conf on Indexer(s)/Heavy Forwarder(s) whichever comes first.

[YourSourceTypeHere]
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+)(?=\{\"userDetails\"\:)
..other timestamp extraction attributes...
0 Karma

ddrillic
Ultra Champion

Gorgeous as usual ; -)
But, any way to avoid the hard-coding of userDetails?

Needless to say - working as expected !!!!!!!!!!

0 Karma

somesoni2
SplunkTrust
SplunkTrust

Well, you generally need to put an anchor for identifying line start. You can try with ([\r\n]+)(?=\{\"\w+\"\:) to see if that works for. Since we don't have full events, we can't say for sure that it'll work (there may be other entries matching that pattern).

DavidHourani
Super Champion

Hi ddrillic,

This usually happens when you have brackets at the beginning of your JSON containing the entire document. It makes it as if the entire document is a value for one of the elements. You should set up a sedcmd in your props to clear this up, or clear it via script before the data gets into Splunk.

If you post a copy of the header/end of your JSON file I can help you set up the sedcmd.

Regards,
David

0 Karma

ddrillic
Ultra Champion

Interesting - it looks like {"userDetails":{...."message":null} followed by another one like this one - {"userDetails":{...."message":null}...

0 Karma

DavidHourani
Super Champion

if your lines are always starting with a new element you can go for this config :

[yourSourcetype]
BREAK_ONLY_BEFORE = ^\{
0 Karma

skoelpin
SplunkTrust
SplunkTrust

LINE_BREAKER would be a much better approach than BREAK_ONLY_BEFORE

0 Karma

DavidHourani
Super Champion

why do you say that ?

0 Karma

skoelpin
SplunkTrust
SplunkTrust

If you set SHOULD_LINEMERGE = false and use LINE_BREAKER, this will skip the merging pipeline and give a performance boost

http://wiki.splunk.com/Community:HowIndexingWorks

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

Hi @ddrillic,

Can you please provide some sample data?

0 Karma

niketn
Legend

@ddrillic also add what is your current sourcetype stanza for JSON data?

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

ddrillic
Ultra Champion

@niketnilay, sorry for the delay. We didn't set anything in the configuration files.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...