Hello,
On a Splunk dashboard, Is there a way to show when data was ingested, stored, and analyzed? I'm trying to build a dashboard to show the following:
I want to prove that it all happened within 10 minutes, and then show that proof in a dashboard. I'm using Splunk Enterprise Search and reporting.
Could you please help me write a query for that or if there are any examples that would be great. Thank you for help in advance!
Source time is your _time
field in the index
Ingested time/arrival time/fetched time all should be same that is when the data got ingested it is available for search- the field is _indextime
Source time is your _time
field in the index
Ingested time/arrival time/fetched time all should be same that is when the data got ingested it is available for search- the field is _indextime
This is what i have so far, however, the time is in seconds now, and i don't understand what that time means. Is there a way to change the seconds to (H:M:S). Again thank you for help!
Source = " " index= " " host = " "
| eval delay_sec=_indextime-_time
| timechart min(delay_sec) avg(delay_sec) max(delay_sec) by source
The value delay_sec should be in seconds , you don't need to convert it in H:M:S format. If you want to display in minutes, you can divide it by 60
I tried dividing by 60 it didn't show any results.
what is the value it shows you before dividing by 60?
Ideally you should be getting the delay in seconds and you do not need any conversion if the delay is usually a few seconds.
Here are all the values i'm getting after running this query:
Source = " " index= " " host = " "
| eval delay_sec=_indextime-_time
| timechart min(delay_sec) avg(delay_sec) max(delay_sec) by source
avg(delay_sec): 19720226
max(delay_sec): 19936226
min(delay_sec): 19504226
I'm trying to check how long did this process takes after ingested data, and when it shows on my search by having a dashboard. But the values above is not really telling me much. Is there a better way ? Sorry for asking a lot of questions
Use the query like this-
Source = " " index= " " host = " "
| eval delay_sec=_indextime-_time
| timechart min(delay_sec) as min_delay avg(delay_sec) as avg_delay max(delay_sec) as max_delay by source
| eval avg_delay=strftime(avg_delay, "%Y-%m-%d %H:%M:%S")
| eval min_delay=strftime(avg_delay, "%Y-%m-%d %H:%M:%S")
| eval max_delay=strftime(avg_delay, "%Y-%m-%d %H:%M:%S")
I got the same exact values in seconds still
You actually dont need to use strftime. The delay is the timedifference in seconds between the index time and the sourcetime.
By looking at the values of result it looks like there is a huge delay as 19720226 seconds which is around 350 days. Can you share the value of _time from your logs
_time: 2018-01 I'm testing old data but this is just for testing purpose and learning. I'm just trying to see if it's possible to do it the way i have it already.
Also, min, avg, max what does those exactly tell me in this case?
Well min here tells you minimum difference between indextime and _time for a particular source, avg will give you an average delay, and max will give you maximum delay between index time and _time field for a given source.
Also the value of _time depends on the datetimestamp configured for your logs. Ideally it should be date timestamp inside your logs but sometimes it is set as Current date and time in which case index time and _time would be same.
The best way to check the different between a set of ecents would be just to display logs with _time and _indextime value
index = <yourindex> sourcetype=<yoursourcetype>| eval time=strftime(_time,"%Y-%m-%d %H:%M:%S")| eval indextime=strftime(_indextime,"%Y-%m-%d %H:%M:%S")| table _raw time indextime
Thank you!