Can Splunk interpret json?

[nsxdavid]

I have event files in json format. Splunk doesn't seem to know to make of it. Is this outside of Splunk's capabilities?

[southeringtonp · Accepted solution]

Please see gkanapathy's comment and answer regarding updates since Splunk version 4.3

---

JSON (JavaScript Object Notation) is a structured data format - there's an example here: http://www.json.org/example.html

Splunk will be able to index the full JSON message without any difficulty, since it's just ascii text. Getting full meaning out of it may take some work.

If you just need text or the occasional specific field, then you can do this with simple expressions and the rex command.

For more complete support there are a few options, including:

Have Splunk read the JSON input via a scripted input, and have the script flatten the JSON data to a better supported form (tabular, CSV, XML, ...)

Create a search command to flatten JSON to XML, and then pipe the result to xmlkv

Create your own JSONkv command to function similarly to xmlkv. Python already supports JSON, so it should not be outrageously difficulty. You may also be able to get ideas from how the XML is handled in $SPLUNK_HOME/etc/apps/search/bin/xmlkv.py

[gkanapathy · Accepted solution]

Answers prior to the release of Splunk 4.3 in January 2012 are somewhat out of date. In that release, indexed JSON can now be extracted as structured JSON fields, either automatically via a new KV_MODE = json setting, or on-demand using the new spath search command. There is no need to create a new search command, and you don't need to flatten or reformat the input.

[ziegfried]

There is an app which add a jsonkv command to Splunk which acts like xmlkv for JSON formatted data. Check it out:

http://splunk-base.splunk.com/apps/22337/jsonutils

[chackmann]

works great for me! - only part im missing is using it directly on the input streams as a REPORT/TRANSFORM rule thing directly in my props.conf =(

[southeringtonp · Accepted solution]

Please see gkanapathy's comment and answer regarding updates since Splunk version 4.3

---

JSON (JavaScript Object Notation) is a structured data format - there's an example here: http://www.json.org/example.html

Splunk will be able to index the full JSON message without any difficulty, since it's just ascii text. Getting full meaning out of it may take some work.

If you just need text or the occasional specific field, then you can do this with simple expressions and the rex command.

For more complete support there are a few options, including:

Have Splunk read the JSON input via a scripted input, and have the script flatten the JSON data to a better supported form (tabular, CSV, XML, ...)

Create a search command to flatten JSON to XML, and then pipe the result to xmlkv

Create your own JSONkv command to function similarly to xmlkv. Python already supports JSON, so it should not be outrageously difficulty. You may also be able to get ideas from how the XML is handled in $SPLUNK_HOME/etc/apps/search/bin/xmlkv.py

[gkanapathy]

There is more to say as of the release of Splunk 4.3 in January 2012. In that release, indexed JSON can now be extracted as structured JSON fields, either automatically via a new KV_MODE = json setting, or on-demand using the new spath search command. There is no need to create a new search command, and you don't need to flatten or reformat the input.

[nsxdavid]

So basically I opted to transform the JSON into key=value pairs prior to delivery to a raw TCP input.

Seems key=value works fine, but "key" : "value" is not readily understood. Or, more likely, I just don't know what I'm doing quite yet. But no matter, this approach is simple, lighter weight, and... well... done. 🙂

[Branden]

I believe Splunk supports jQuery, which supports JSON. Have you looked into the jQuery.getJSON() function? I've never used this in the context of Splunk, however, so I may be way off.

[ziegfried]

jQuery runs on the client side. So it won't help much in terms of processing JSON formatted events.

[Christian]

How does this Format look like ? Can you give us some examples ? There are several possibility's to "teach" splunk. A few more Informations would help