Is there easy way to find if the data is in future. We have ingested data into splunk, but don't see them in dashboard. We are looking for easy way to identify such data.
As per the comment from skoelpin you can either use the advanced settings or in your search add:
earliest=+5m latest=+10y
Or similar...
In the app Alerts For Splunk Admins I have an alert called "IndexerLevel - Future Dated Events that appeared in the last week", for this exact purpose.
If you want just the one search it's in github here
You can simply select the timerange picker and select Advanced
.. Earliest=+1s Latest=+2mon
Or if you wanted to do it all in SPL, you could do this
index=... earliest=+1s latest=+2mon
Here is a test I did. Created a sample test.log file with data for 2019
64.242.88.10 - - [07/Mar/2019:16:58:54 -0800] "GET /mailman/listinfo/administration HTTP/1.1" 200 6459
lordgun.org - - [07/Mar/2019:17:01:53 -0800] "GET /razor.html HTTP/1.1" 200 2869
64.242.88.10 - - [07/Mar/2019:17:09:01 -0800] "GET /twiki/bin/search/Main/SearchResult?scope=text®ex=on&search=Joris%20*Benschop[^A-Za-z] HTTP/1.1" 200 4284
64.242.88.10 - - [07/Mar/2019:17:10:20 -0800] "GET /twiki/bin/oops/TWiki/TextFormattingRules?template=oopsmore¶m1=1.37¶m2=1.37 HTTP/1.1" 200 11400
64.242.88.10 - - [07/Mar/2019:17:13:50 -0800] "GET /twiki/bin/edit/TWiki/DefaultPlugin?t=1078688936 HTTP/1.1" 401 12846
64.242.88.10 - - [07/Mar/2019:17:16:00 -0800] "GET /twiki/bin/search/Main/?scope=topic®ex=on&search=^g HTTP/1.1" 200 3675
64.242.88.10 - - [07/Mar/2019:17:17:27 -0800] "GET /twiki/bin/search/TWiki/?scope=topic®ex=on&search=^d HTTP/1.1" 200 5773
lj1036.inktomisearch.com - - [07/Mar/2019:17:18:36 -0800] "GET /robots.txt HTTP/1.0" 200 68
lj1090.inktomisearch.com - - [07/Mar/2019:17:18:41 -0800] "GET /twiki/bin/view/Main/LondonOffice HTTP/1.0" 200 3860
64.242.88.10 - - [07/Mar/2019:17:21:44 -0800] "GET /twiki/bin/attach/TWiki/TablePlugin HTTP/1.1" 401 12846
64.242.88.10 - - [07/Mar/2019:17:22:49 -0800] "GET /twiki/bin/view/TWiki/ManagingWebs?rev=1.22 HTTP/1.1" 200 9310
64.242.88.10 - - [07/Mar/2019:17:23:54 -0800] "GET /twiki/bin/statistics/Main HTTP/1.1" 200 808
64.242.88.10 - - [07/Mar/2019:17:26:30 -0800] "GET /twiki/bin/view/TWiki/WikiCulture HTTP/1.1" 200 5935
Today is 03/29/2018 17:26:11
After that data is indexed we need to check if the data is in future.
source="*test.log.log" host="test" sourcetype="apache" | eval delay=_indextime-_time | convert ctime(_time) AS xtime |convert ctime(_indextime) AS indextime | table _raw delay _time indextime xtime
It shows that data is early by 773981.
We can see same information in metrics.log
03-29-2018 17:26:12.099 -0700 INFO Metrics - group=per_sourcetype_thruput, series="test", kbps=0.1403387022431233, eps=1.258039598828591, kb=4.3505859375, ev=39, avg_age=776259.9487179487, max_age=778822
03-29-2018 17:26:12.098 -0700 INFO Metrics - group=per_host_thruput, series="test", kbps=0.1403387022431233, eps=1.258039598828591, kb=4.3505859375, ev=39, avg_age=776259.9487179487, max_age=778822