Getting Data In

Assign sequential timestamps to data on data load

NK_1
Path Finder

I tried loading a textfile (via "splunk add oneshot datafile.txt") containing just IP addresses into Splunk v5.0, and Splunk tried to assign some really strange timestamps (2010 - 2019?) to some of those IP Address lines.

I suspect it tried to interpret some of the the ip address numbers as dates:


$ grep DateParserVerbose ./var/log/splunk/splunkd.log | grep 2019

07-16-2013 17:31:16.085 -0700 WARN DateParserVerbose - Accepted time (Sun Nov 24 19:11:24 2019) is suspiciously far away from the previous event's time (Sun Nov 21 12:33:24 2010), but still accepted because it was extracted by the same pattern. Context: source::(filename.csv)|host::(hostname)|(filename)|

07-16-2013 17:31:16.085 -0700 WARN DateParserVerbose - A possible timestamp match (Sun Nov 24 19:11:24 2019) is outside of the acceptable time window. If this timestamp is correct, consider adjusting MAX_DAYS_AGO and MAX_DAYS_HENCE. Context: source::(filename.csv)|host::(hostname)|(filename)|

Is there a way to make Splunk assign sequential timestamps and not try to parse a timestamp from the data?

Tags (3)
0 Karma

wagnerbianchi
Splunk Employee
Splunk Employee

If Splunk does not find a timestamp for each line contained into a file, the timestamp of the file creation will be used asvthe event timestamp. What is the timestamp format of your operation system? Tks!

0 Karma

NK_1
Path Finder

I don't recall specifying or overriding any default for the timestamp format, but here's some info:


$ echo $OSTYPE
linux-gnu

$ date
Mon Jul 22 15:01:50 PDT 2013

0 Karma

NK_1
Path Finder

I ended up doing this to prevent Splunk from guessing timestamps for now, but I'm looking for a better solution:

# add timestamp and key to every line in text file 
cat datafile.txt | xargs -d"\n" -I {} date +"%Y.%m.%d %H:%M:%S.%N ip={}" > datafile1.txt

0 Karma
Get Updates on the Splunk Community!

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...