Getting Data In

JSON time field not parsed correctly

wsw70
Communicator

Hello

I submit files with JSON-encoded lines to splunk, to a monitored directory. The fields are extracted correctly, except for the time which is not.

My props.conf file:

[nessusjson]
INDEXED_EXTRACTIONS = json
KV_MODE = none
TIMESTAMP_FIELDS = 'N_scantime'
TIME_FORMAT = '%s'
MAX_TIMESTAMP_LOOKAHEAD = -1

I also tried KV_MODE = json.

Below is an example of a JSON line, as seen by splunk. The EPOCH in N_scantime corresponds to Thu, 27 Nov 2014 09:17:08 GMT.

11/27/14 
3:22:38.000 PM  
{ [-] 
   N_exploit: false 
   N_exploit_malware: UNKNOWN 
   N_exploit_metasploit: UNKNOWN 
   N_exploit_metasploit_name: UNKNOWN 
   N_nettype: wazaatype 
   N_scantime: 1417079828 
   N_subnetname: wazaa name 
   N_timeduration: 680 
   N_timeend: Thu Nov 27 14:33:58 2014 
   N_timeend_epoch: 1417098838 
   N_timestart: Thu Nov 27 14:22:38 2014 
   N_timestart_epoch: 1417098158 
   N_vendor: java 
}

The idea was to have a specific field ( N_scantime ) in EPOCH format and other informational fields with the date ( N_time...).
I think I put in props.conf everything to indicate the field ( TIMESTAMP_FIELDS - I tried both with quotes and without), the format ( TIME_FORMAT - which is a 10 digits EPOCH in my case) and the fact that the filed can be anywhere ( MAX_TIMESTAMP_LOOKAHEAD). I restarted the server for each test.

It looks like the N_timestart field was parsed instead.

Is there something I am still missing?

Thank you for any pointers!

0 Karma
1 Solution

wsw70
Communicator

OK, I found what was wrong: I forgot to link the entry in props.conf with a source... Specifically I forgot to add

sourcetype = nessusjson

to the relevant entry which describes my input in inputs.conf

I will leave the question and answer, in case someone stumbles upon one day, searching for time extraction in JSON.

View solution in original post

wsw70
Communicator

OK, I found what was wrong: I forgot to link the entry in props.conf with a source... Specifically I forgot to add

sourcetype = nessusjson

to the relevant entry which describes my input in inputs.conf

I will leave the question and answer, in case someone stumbles upon one day, searching for time extraction in JSON.

Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...