Getting Data In

JSON time field not parsed correctly

wsw70
Communicator

Hello

I submit files with JSON-encoded lines to splunk, to a monitored directory. The fields are extracted correctly, except for the time which is not.

My props.conf file:

[nessusjson]
INDEXED_EXTRACTIONS = json
KV_MODE = none
TIMESTAMP_FIELDS = 'N_scantime'
TIME_FORMAT = '%s'
MAX_TIMESTAMP_LOOKAHEAD = -1

I also tried KV_MODE = json.

Below is an example of a JSON line, as seen by splunk. The EPOCH in N_scantime corresponds to Thu, 27 Nov 2014 09:17:08 GMT.

11/27/14 
3:22:38.000 PM  
{ [-] 
   N_exploit: false 
   N_exploit_malware: UNKNOWN 
   N_exploit_metasploit: UNKNOWN 
   N_exploit_metasploit_name: UNKNOWN 
   N_nettype: wazaatype 
   N_scantime: 1417079828 
   N_subnetname: wazaa name 
   N_timeduration: 680 
   N_timeend: Thu Nov 27 14:33:58 2014 
   N_timeend_epoch: 1417098838 
   N_timestart: Thu Nov 27 14:22:38 2014 
   N_timestart_epoch: 1417098158 
   N_vendor: java 
}

The idea was to have a specific field ( N_scantime ) in EPOCH format and other informational fields with the date ( N_time...).
I think I put in props.conf everything to indicate the field ( TIMESTAMP_FIELDS - I tried both with quotes and without), the format ( TIME_FORMAT - which is a 10 digits EPOCH in my case) and the fact that the filed can be anywhere ( MAX_TIMESTAMP_LOOKAHEAD). I restarted the server for each test.

It looks like the N_timestart field was parsed instead.

Is there something I am still missing?

Thank you for any pointers!

0 Karma
1 Solution

wsw70
Communicator

OK, I found what was wrong: I forgot to link the entry in props.conf with a source... Specifically I forgot to add

sourcetype = nessusjson

to the relevant entry which describes my input in inputs.conf

I will leave the question and answer, in case someone stumbles upon one day, searching for time extraction in JSON.

View solution in original post

wsw70
Communicator

OK, I found what was wrong: I forgot to link the entry in props.conf with a source... Specifically I forgot to add

sourcetype = nessusjson

to the relevant entry which describes my input in inputs.conf

I will leave the question and answer, in case someone stumbles upon one day, searching for time extraction in JSON.

Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...