Splunk Search

How to automatically extract the JSON object before indexing so I don't have to use spath in my search?

Kukkadapu
Path Finder

Hi,

How do I extract the JSON object before indexing itself? Right now I'm extracting using the below search.

This is the data:

2016-01-18 16:24:40,406 INFO  [org.apache.log4j.Logger] (ajp-/10.32.20.21:8309-7) 
transaction_id="123451" 
action="ABC API" 
desc="start of api" 
result="success" 
http_method="POST"
payload_json=
{
    "requestId": "ABCDEqq",
    "partnerId": "001",
    "storeId": "001",
    "subscriberId": "001",
    "event": "1",
    "date": "2015-10-20 12:08:56 PDT",
    "uuid": "123451"
}

Here is the search:

index="xyz" sourcetype="pm" action="ABC API" | spath input=payload_json  | stats count by action,event

It works fine, but is there a way to extract the JSON before indexing itself so the search is going to be:

index="xyz" sourcetype="pm" action="ABC API" | stats count by action,event 

No spath in this command

0 Karma
1 Solution

s2_splunk
Splunk Employee
Splunk Employee

If your payload_json is pretty static, you could create calculated fields using spath(payload_json, "requestId") (for example) as the eval expression. It would still execute at search time, though. There probably is a way to write a generic EXTRACT/TRANSFORM as well to dynamically get the fields extracted from the payload_json field you already have.

View solution in original post

s2_splunk
Splunk Employee
Splunk Employee

If your payload_json is pretty static, you could create calculated fields using spath(payload_json, "requestId") (for example) as the eval expression. It would still execute at search time, though. There probably is a way to write a generic EXTRACT/TRANSFORM as well to dynamically get the fields extracted from the payload_json field you already have.

Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...