Getting Data In

Json Indexing

nsiva23
Explorer

We are trying parse a Json file for indexing. While parsing we have two events in the json file mentioned below

[
{
"timestamp": "2014-04-07 00:00:36.297",
"source": "HOST",
"sourcetype": "Windows",
"host": "10.10.10.10",
"hardwareRevision": "0200"
},
{
"timestamp": "2014-04-07 00:00:36.297",
"source": "HOST",
"sourcetype": "LINUX",
"host": "10.10.10.20",
"hardwareRevision": "NA"
}
]

I am declaring the following params in props.conf

KV_MODE=json
NO_BINARY_CHECK=1
TIME_FORMAT=%Y-%m-%d %H:%M:%S.%3N

But indexer recognizes only one event instead of two. It doesn't recognizes second time stamp. I don't see any examples of props.conf for Json on the internet. Is there is any samples of props.conf how it should look like for the Json file.

Thanks in advance.

Tags (2)

fdi01
Motivator

Since you have multiple time stamps in the event, you need select the right one for Splunk to recognize.

http://docs.splunk.com/Documentation/Splunk/6.2.2/Data/Configuretimestamprecognition

Your format should be the :

timestamp": "2014-04-07 00:00:36.297",

Portion from your json.

So in your props, your timestamp format should be something like:

TIME_FORMAT=%Y-%m-%d %H:%M:%S.%3N
 TIME_PREFIX= \t

I'm not sure if there is a tab, you could also try

TIME_PREFIX = (AM\s|PM\s)

That looks for AM OR PM (not its followed by a space. That might be a tab in your json..)

hkeebler
Engager

Thanks so much for the response. This still doesn't work for me. My sample data is below. I don't really care if any of the dates are used as timestamps. Today's date as default would be fine. I just want separate events to be recognized.

{"respId":"search","status":"0","events":[{"name":"event1","startDate":"20150404:1600",...move stuff,"dateReceived":"20150405:0000"},{"name":"event2",.........}],"count":"500"}

If I use the default _json (Indexed_extractions = json) I end up with fields respId, status, events{}:name, events{}:startDate, events{}:dateReceived etc. where all the data for these fields is now crammed into the one field - i.e., one event.
My props.conf fields are
INDEXED_EXTRACTIONS = json
TIME_FORMAT = %Y%m%d:%H%M
BREAK_ONLY_BEFORE = {"name"
TIME_PREFIX = dateReceived":

0 Karma

hkeebler
Engager

Did anyone ever answer your question? I have the same issue now. I have something that is all json and includes a json array "outages: [{event1},{event2},....] ". I cannot get it to recognize separate events, even with the BREAK_ONLY_BEFORE = clause.

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to May Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars this month. This ...

They're back! Join the SplunkTrust and MVP at .conf24

With our highly anticipated annual conference, .conf, comes the fez-wearers you can trust! The SplunkTrust, as ...

Enterprise Security Content Update (ESCU) | New Releases

Last month, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...