Getting Data In

JSON data with date not indexed

osec2a
New Member

Hi,

I am trying to index JSON data but Splunk refused to index it and I have no errors in logs.
The format of my data is :

{"test1":"2017-08-31", "test2":"12.34.56"}

I tried many format :

{"test1":"2017-08-31", "test2":"12.34.56"}" => not indexed
{"test1":"12.34.56", "test2":"2017-08-31"}" => not indexed
{"test1":"2017-08-31", "test2":2017/08/3"}"  = > indexed successfully
{"test1":"2017.08.31", "test2":17.08.3"}" => indexed successfully
{"test1":"2017_08_31", "test2":17.08.3"}"  => indexed successfully

Why Splunk doesn't want to index my JSON with these data :

{"test1":"2017-08-31","test2":"12.34.56"}

Thanks for your help

0 Karma

cpetterborg
SplunkTrust
SplunkTrust

If you explicitly set the data format to be %Y-%m-%d, does it work in all cases? (props.conf) I suspect that by default Splunk is probably looking at 12.34.56 as a date string, and if it falls outside the range of the date you are looking at, or it is prior to 1970 (I'm assuming that 12.34.56 is a made up piece of data, but I have nothing else to go on), then you are not going to find it in your search or it is too old to be properly indexed.

Also, though it is probably just a typo, your JSON data in your example data is bogus JSON. It probably should look like this:

{"test1":"2017-08-31", "test2":"12.34.56"} => not indexed
{"test1":"12.34.56", "test2":"2017-08-31"} => not indexed
{"test1":"2017-08-31", "test2":"2017/08/3"}  = > indexed successfully
{"test1":"2017.08.31", "test2":"17.08.3"} => indexed successfully
{"test1":"2017_08_31", "test2":"17.08.3"}  => indexed successfully
0 Karma

niketn
Legend

@osec2a, I tried adding {"test1":"2017-08-31", "test2":"12.34.56"} and it worked fine for me.

I had done Single File Upload in preview mode and I just created my Custom Sourcetype under "Search & Reporting App" with index="main".

The data looked good in preview mode as well as after adding to index in Splunk Search.

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma

osec2a
New Member

My data are outputed by python script.
My script output is json data.
When the script output is {"test1":"2017-08-31", "test2":"12.34.56"}, data are not indexed, but when I delete the dot ".", data are indexed.

0 Karma
Get Updates on the Splunk Community!

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...