Splunk Search

Can Splunk interpret json?

nsxdavid
Engager

I have event files in json format. Splunk doesn't seem to know to make of it. Is this outside of Splunk's capabilities?

2 Solutions

southeringtonp
Motivator

Please see gkanapathy's comment and answer regarding updates since Splunk version 4.3


JSON (JavaScript Object Notation) is a structured data format - there's an example here: http://www.json.org/example.html

Splunk will be able to index the full JSON message without any difficulty, since it's just ascii text. Getting full meaning out of it may take some work.

If you just need text or the occasional specific field, then you can do this with simple expressions and the rex command.

For more complete support there are a few options, including:

  • Have Splunk read the JSON input via a scripted input, and have the script flatten the JSON data to a better supported form (tabular, CSV, XML, ...)

  • Create a search command to flatten JSON to XML, and then pipe the result to xmlkv

  • Create your own JSONkv command to function similarly to xmlkv. Python already supports JSON, so it should not be outrageously difficulty. You may also be able to get ideas from how the XML is handled in $SPLUNK_HOME/etc/apps/search/bin/xmlkv.py

    View solution in original post

    gkanapathy
    Splunk Employee
    Splunk Employee

    Answers prior to the release of Splunk 4.3 in January 2012 are somewhat out of date. In that release, indexed JSON can now be extracted as structured JSON fields, either automatically via a new KV_MODE = json setting, or on-demand using the new spath search command. There is no need to create a new search command, and you don't need to flatten or reformat the input.

    View solution in original post

    gkanapathy
    Splunk Employee
    Splunk Employee

    Answers prior to the release of Splunk 4.3 in January 2012 are somewhat out of date. In that release, indexed JSON can now be extracted as structured JSON fields, either automatically via a new KV_MODE = json setting, or on-demand using the new spath search command. There is no need to create a new search command, and you don't need to flatten or reformat the input.

    ziegfried
    Influencer

    There is an app which add a jsonkv command to Splunk which acts like xmlkv for JSON formatted data. Check it out:

    http://splunk-base.splunk.com/apps/22337/jsonutils

    chackmann
    Engager

    works great for me! - only part im missing is using it directly on the input streams as a REPORT/TRANSFORM rule thing directly in my props.conf =(

    0 Karma

    southeringtonp
    Motivator

    Please see gkanapathy's comment and answer regarding updates since Splunk version 4.3


    JSON (JavaScript Object Notation) is a structured data format - there's an example here: http://www.json.org/example.html

    Splunk will be able to index the full JSON message without any difficulty, since it's just ascii text. Getting full meaning out of it may take some work.

    If you just need text or the occasional specific field, then you can do this with simple expressions and the rex command.

    For more complete support there are a few options, including:

  • Have Splunk read the JSON input via a scripted input, and have the script flatten the JSON data to a better supported form (tabular, CSV, XML, ...)

  • Create a search command to flatten JSON to XML, and then pipe the result to xmlkv

  • Create your own JSONkv command to function similarly to xmlkv. Python already supports JSON, so it should not be outrageously difficulty. You may also be able to get ideas from how the XML is handled in $SPLUNK_HOME/etc/apps/search/bin/xmlkv.py

    gkanapathy
    Splunk Employee
    Splunk Employee

    There is more to say as of the release of Splunk 4.3 in January 2012. In that release, indexed JSON can now be extracted as structured JSON fields, either automatically via a new KV_MODE = json setting, or on-demand using the new spath search command. There is no need to create a new search command, and you don't need to flatten or reformat the input.

    0 Karma

    nsxdavid
    Engager

    So basically I opted to transform the JSON into key=value pairs prior to delivery to a raw TCP input.

    Seems key=value works fine, but "key" : "value" is not readily understood. Or, more likely, I just don't know what I'm doing quite yet. But no matter, this approach is simple, lighter weight, and... well... done. 🙂

    0 Karma

    Branden
    Builder

    I believe Splunk supports jQuery, which supports JSON. Have you looked into the jQuery.getJSON() function? I've never used this in the context of Splunk, however, so I may be way off.

    0 Karma

    ziegfried
    Influencer

    jQuery runs on the client side. So it won't help much in terms of processing JSON formatted events.

    0 Karma

    Christian
    Path Finder

    How does this Format look like ? Can you give us some examples ? There are several possibility's to "teach" splunk. A few more Informations would help

    0 Karma
    Get Updates on the Splunk Community!

    Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

    March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

    What’s New in Splunk App for PCI Compliance 5.3.1?

    The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

    Extending Observability Content to Splunk Cloud

    Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...