Splunk Search

I would like to index and make the KV that are in the JSON available as searchable fields

andrewbeak
Path Finder

I am using Splunk Cloud which means I do not have access to the server.

I have log lines that look like this:

Jan  2 18:32:25 hostname analyticsEvent[32152]: {"message":"{"userId":"72",},"extra":[]}

In other words, there is a JSON encoded string in the line.

I would like to index and make the KV that are in the JSON available as searchable fields.

How is this possible?

0 Karma
1 Solution

micahkemp
Champion

To remove non-json from your events, so that the JSON is automatically handled at search type, put this in props.conf:

[<sourcetype name>]
SEDCMD-keepjson = s/^[^{]*{/{/

Note: this will alter _raw at index time. So be sure to test this in a dev system before making this change in production.

View solution in original post

micahkemp
Champion

To remove non-json from your events, so that the JSON is automatically handled at search type, put this in props.conf:

[<sourcetype name>]
SEDCMD-keepjson = s/^[^{]*{/{/

Note: this will alter _raw at index time. So be sure to test this in a dev system before making this change in production.

andrewbeak
Path Finder

Thanks, this looks like what I need. I'm using Splunk Cloud and from what I read the props.conf is set up on the server (which I don't have access to). Do you know how to change it on Splunk Cloud?

0 Karma

micahkemp
Champion

Based on previous answers post:

<your base search>
 | rex "(?<json>\{.+)" 
 | spath input=json

andrewbeak
Path Finder

Thanks for the answer, but I'm hoping to have the keys indexed so that they appear on the left side of the search panel to allow filtering and reporting on their values. I'd prefer to avoid having to use search modifiers as much as possible (not all of my users are programmers). Is there no way to edit the sourcetype to tell it "ignore this pattern of characters and then json_decode the rest"

0 Karma

micahkemp
Champion

You could rewrite _raw at index time. That would happen after timestamping, so it might be a viable solution for you. Your props.conf might look like:

[<sourcetype>]
SEDCMD-keepjson = s/^[^{]*//

Edit: fixed SEDCMD line. Added answer with this info, since it's a different solution than the answer of which this comment is a child.

0 Karma

micahkemp
Champion

I'm refining this response. It doesn't appear to work as currently stated.

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to May Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars this month. This ...

They're back! Join the SplunkTrust and MVP at .conf24

With our highly anticipated annual conference, .conf, comes the fez-wearers you can trust! The SplunkTrust, as ...

Enterprise Security Content Update (ESCU) | New Releases

Last month, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...