Splunk Search

what is apiStartTime='ZERO_TIME'

sansay
Contributor

I have been investigating excessively expensive searches by querying the audit log, and I came across one that has this time range:
apiStartTime='ZERO_TIME', apiEndTime='ZERO_TIME'

Anyone knows what this means?

Tags (3)

dvg06
Path Finder

These could be real time searches.
I ran a search like "index=*" for 30 seconds realtime, and the apiStartTime was displayed as Zero_time

search total_run_time _time apiStartTime apiEndTime search_type user
search index=* 2018-03-20 10:28:09.913 ZERO_TIME ZERO_TIME ad hoc test_user01
search index=* 2018-03-20 10:28:13.560 ZERO_TIME ZERO_TIME ad hoc test_user01

0 Karma

lguinn2
Legend

The audit log captures the time range of the search. As a Splunk user, you specify the time range by using the pull-down menu (or by using the earliest and latest keywords). When Splunk processes the search, it calculates the actual time that should be searched. apiStartTime represents the earliest time, and apiEndTime represents the latest time.

EDIT - in my original answer, I said

apiStartTime='ZERO_TIME', apiEndTime='ZERO_TIME' means that the search ran over All Time. It makes sense that this would be an excessively expensive search.

but this appears not to be the case.

END EDIT

0 Karma

lguinn2
Legend

No problem - please post if you figure it out...

0 Karma

sansay
Contributor

Sorry but, indeed, it seems that your original answer is wrong.
A simpler search, without apiStartTime='ZERO_TIME' apiEndTime='ZERO_TIME', returns a bunch of other records, including the very same query, with the exact time range selected by the user. And this query occured just microseconds before the one with ZERO_TIME. So it must be something splunk does, but because it happens all the time it can't mean that it's the "All time" time range that was used.
So I have to remove the point. I will add this in a splunk ticket I opened to resolve cold storage searches that take our system down.

0 Karma

sarnagar
Contributor

@sansay ,
Could you please let me know wht this actually means if you are aware of it now?

apiStartTime='ZERO_TIME', apiEndTime='ZERO_TIME'

0 Karma

sansay
Contributor

Sorry but no, I haven't figured it out. I haven't had the time to even think about this issue.

0 Karma

lguinn2
Legend

Perhaps I am wrong. Could this have been something run by Splunk internally?

0 Karma

sansay
Contributor

This gets weirder and weirder, according to my last search, and if apiStartTime='ZERO_TIME', apiEndTime='ZERO_TIME' means "All time", even I ran "All time" queries. This is starting to sound more and more like a bug.

0 Karma

sansay
Contributor

Thank you very much lguinn.
The weird thing is that I disabled the "All time" from the GUI. And the user, from being the previous Splunk admin knows very well not to run "All time" queries. And he confirmed that when asked. So how else could this happen?

Is there any way I can get the exact query that was executed, ie, with the time range specified by the user?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...