Splunk Search

How to extract fields from a json in _raw?

Adisharma
Engager

I have an application which logs data in the following form:

2023-06-30T12:21:08Z DEBUG scalehandler Getting metrics from scaler {"scaledObject.Namespace": "my-namespace", "scaledObject.Name": "my-app", "scaler": "myScaler", "metricName": "http_count_total", "metrics": [], "scalerError": "error getting metrics"}

From the above _raw, I want to extract fields such as scaledObject.Name, scaler and scalerError which I would then use to create an alert.

Could someone help me in creating new fields for the above mentioned json fields using the rex function? 

Labels (3)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

The easiest (although maybe not the most effective) solution would be to use regex to capture the json part and then use spath to extract fields from this part.

<your_search>
| rex "(?<json>\{.*\})"
| spath input=json

 (I'm not sure if the curly braces need escaping or not).

isoutamo
SplunkTrust
SplunkTrust

One comment for this workaround. It extracts those fields as expected, BUT if any event is shorter than 10241 (or whatever you have in limits.conf/kv stanza) character then you have duplicate fields on those events!

Basically you can try to remove duplicates e.g. 

| mvexpand <field name>
| dedup <field name>

Usually this must done one field only. Anyhow this is just workaround until you can fix those values in limits.conf (kv stanza).

See e.g. https://community.splunk.com/t5/Splunk-Search/Why-are-not-all-field-values-are-extracted-for-long-JS... 

0 Karma

dural_yyz
Communicator

https://community.splunk.com/t5/Splunk-Search/How-to-extract-fields-from-JSON-data-in-Splunk/m-p/274...

Look at the answer from javiergn for the solution I would have chosen.

 

Get Updates on the Splunk Community!

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Wednesday, May 29, 2024  |  11AM PST / 2PM ESTRegister now and join us to learn more about how you can ...

Get Your Exclusive Splunk Certified Cybersecurity Defense Engineer Certification at ...

We’re excited to announce a new Splunk certification exam being released at .conf24! If you’re headed to Vegas ...

Share Your Ideas & Meet the Lantern team at .Conf! Plus All of This Month’s New ...

Splunk Lantern is Splunk’s customer success center that provides advice from Splunk experts on valuable data ...