How to get the depth of one of nested json objects in mixed log line?

[Digister]

Hi guys,

I am stuck on this for hours now, but I can't achieve what I am looking for.
My log lines are looking like this

[2018-08-03T05:14:14.731] [INFO] komvos.controller.scenario - Summary: {"id":"myID","status":"END","scenario":"My Scenario Name","parameters":{"baseUrl":"https://myUrl.com","username":"JohnSmith","isTest":false},"durationInSeconds":"11.614","manualEffortInSeconds":1200,"isCounting":false,"isTest":true,"isStatusEnd":true,"existSuccessfulRunTodayWithSameParameters":false, "result":{"statusCode":"200","type":"myType","description":"myDescription" }

What I want is:

Time     | ID | Scenario | Parameters | Duration in Seconds | Manual Effort in Seconds | isCounting | ... | result

 2018... | myID |My Scenario Name  |  {"baseUrl":"https://myUrl.com","username":"JohnSmith","isTest":false} | 11.614 | 1200 | ... | {"statusCode":"200","type":"myType","description":"myDescription" }

What I did is the follwowing:

index="..."  | rex "^.{71}(?.*)"| spath input=data 

After this search I can access every attribute inside my data object and inside parameters and result, but I can't achieve writing both nested objects in my table.
However the objects parameter and result can greatly defer from one log line to another, also this is just a special log line, most of the log is containing only text.

Any idea how I can achieve this? Every help is highly appreciated.
Thanks,
Fabian

[Digister · Accepted solution]

Hi there,

after reading intensively the spath documentation I found a solution:

I used the following to get the complete parameters jobject:

index="c1006*" "- Summary" | rex "^.{71}(?.*)" | spath input=data| spath input=data path=parameters output=parameters | table _time, id, scenario, status, host, parameters

[coccyx]

Why are you trying to count characters in your regex? This regex seems to do the trick: \s(?{.+})$.

In general, I'd try to get your developers to clean up their logging to output just JSON rather than JSON in a log. Another option to prettify the logs at ingestion time. It's a use case we're working on at Cribl (https://www.cribl.io/) if you're interested.

[Digister]

I am not trying to count characters, I am removing 71 characters to only get the json data as input for spath, (rex "^.{71}(?.*)"). Therefore the regey you posted does not do the trick , but thanks for your comment! Unfortunately it is not going to happen, that they change the log. So my problem is still how can I access nested objects in general using spath?