Getting Data In

how to split multiline JSON events/ Group multiline JSON event.

AKG1_old1
Builder

HI,

Log File

[ {
  "name" : "TraderCurrency",
  "type" : "RiskBreakdown",
  "duration" : 1173,
  "count" : 1,
  "average" : 1173.0
}, {
  "name" : "CounterpartyPreSettlement",
  "type" : "RiskBreakdown",
  "duration" : 312934,
  "count" : 1,
  "average" : 312934.0
}, {
  "name" : "CounterpartyNotional",
  "type" : "RiskBreakdown",
  "duration" : 406789,
  "count" : 1,
  "average" : 406789.0
} ]

I can remove "[ ] ," using sed to make it correct json.

props.conf
[reset_profiler]
SEDCMD-remove_bracket = s/\[|\]//g
SEDCMD-remove_coma = s/\},/}\n/g
KV_MODE = json
NO_BINARY_CHECK = true
BREAK_ONLY_BEFORE  = \{

Thanks

0 Karma
1 Solution

kamlesh_vaghela
SplunkTrust
SplunkTrust

Hi @agoyal,

I have tried with this json and able to make it.

[{"name": "TraderCurrency","type": "RiskBreakdown","duration": 1173,"count": 1,"average": 1173.0}, {"name": "CounterpartyPreSettlement","type": "RiskBreakdown","duration": 312934,"count": 1,"average": 312934.0}, {"name": "CounterpartyNotional","type": "RiskBreakdown","duration": 406789,"count": 1,"average": 406789.0}]

Can you please try this?

props.conf

[reset_profiler]
NO_BINARY_CHECK=true
BREAK_ONLY_BEFORE=([\{])
MUST_BREAK_AFTER=([\},])
SEDCMD-remove_bracket=s/\[|\]//g
LINE_BREAKER=(,\s)
KV_MODE=json

You can extract events like below:

alt text

View solution in original post

kamlesh_vaghela
SplunkTrust
SplunkTrust

Hi @agoyal,

I have tried with this json and able to make it.

[{"name": "TraderCurrency","type": "RiskBreakdown","duration": 1173,"count": 1,"average": 1173.0}, {"name": "CounterpartyPreSettlement","type": "RiskBreakdown","duration": 312934,"count": 1,"average": 312934.0}, {"name": "CounterpartyNotional","type": "RiskBreakdown","duration": 406789,"count": 1,"average": 406789.0}]

Can you please try this?

props.conf

[reset_profiler]
NO_BINARY_CHECK=true
BREAK_ONLY_BEFORE=([\{])
MUST_BREAK_AFTER=([\},])
SEDCMD-remove_bracket=s/\[|\]//g
LINE_BREAKER=(,\s)
KV_MODE=json

You can extract events like below:

alt text

AKG1_old1
Builder

@kamlesh_vaghela : thanks for reply. Are you using all data in single line ?
When i tried getting small issue that it's not recognize as json event because i guess " , " is removed at the end of every value.

Thats what I am getting
{
"name" : "matrixAddonCurrencySwap"
"type" : "Formula"
"duration" : 0
"count" : 15
"average" : 0.0
}

0 Karma

kamlesh_vaghela
SplunkTrust
SplunkTrust

@agoyal,

Yes, I'm using single line data. For multi-line event use below conf.

[reset_profiler]
NO_BINARY_CHECK=true
BREAK_ONLY_BEFORE=([\{])
MUST_BREAK_AFTER=([\},])
SEDCMD-remove_bracket=s/\[|\]//g
LINE_BREAKER=(, )
KV_MODE=json

I'm using following sample:

[{
        "name": "TraderCurrency",
        "type": "RiskBreakdown",
        "duration": 1173,
        "count": 1,
        "average": 1173.0
}, {
        "name": "CounterpartyPreSettlement",
        "type": "RiskBreakdown",
        "duration": 312934,
        "count": 1,
        "average": 312934.0
}, {
        "name": "CounterpartyNotional",
        "type": "RiskBreakdown",
        "duration": 406789,
        "count": 1,
        "average": 406789.0
}]

Happy Splunking

AKG1_old1
Builder

@kamlesh_vaghela : That's Great !! Thank you very much !! 🙂 Happy Splunking

0 Karma

AKG1_old1
Builder

@kamlesh_vaghela: Sorry, I forget to mention one case. one logfile having two blocks. In this case 2nd and 3rd event are not getting seprated. Not sure if possible to use multiple link breaker.

[{
"name": "TraderCurrency",
"type": "RiskBreakdown",
"duration": 1173,
"count": 1,
"average": 1173.0
},{
"name": "TraderCurrency",
"type": "RiskBreakdown",
"duration": 1173,
"count": 1,
"average": 1173.0
} ]
[ {
"name": "CounterpartyPreSettlement",
"type": "Formula",
"duration": 312934,
"count": 1,
"average": 312934.0
}, {
"name": "CounterpartyNotional",
"type": "Formula",
"duration": 406789,
"count": 1,
"average": 406789.0
}]

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...