Getting Data In

Search for events that have not happened

sheltonr
Engager

When we 'fall back' one hour for Daylight Savings Time, I'd like to run a search that would reveal those log sources that didn't adjust correctly (their event timestamps will be one hour in the future.) What might the syntax for such a search look like? Thank you!

Tags (1)
2 Solutions

sideview
SplunkTrust
SplunkTrust

The way I've usually seen this done is to compare _time to _indextime.

_time is the time extracted from the event's text, as the number of seconds since 1970

_indextime is the time the event was indexed, according to the local system clock (also number of seconds since 1970).

so if you use eval to look for this difference you should have what you need.

View solution in original post

araitz
Splunk Employee
Splunk Employee

Ah, the nefarious "future event".

For a lightweight approach, how about:

| metadata type=hosts | eval now = now() | where latestTime > now

View solution in original post

0 Karma

cudgel
Path Finder

How well does the search "| metadata type=hosts | eval now = now() | where latestTime > now" work in a scheduled search? I seem to get a huge load of results for hosts off by as much as 2-3 minutes because of what - now() evaluates to when the search is scheduled to run? If I run the search manually, e.g. over the past hour, it seems to return accurate results.

0 Karma

araitz
Splunk Employee
Splunk Employee

Ah, the nefarious "future event".

For a lightweight approach, how about:

| metadata type=hosts | eval now = now() | where latestTime > now
0 Karma

sideview
SplunkTrust
SplunkTrust

The way I've usually seen this done is to compare _time to _indextime.

_time is the time extracted from the event's text, as the number of seconds since 1970

_indextime is the time the event was indexed, according to the local system clock (also number of seconds since 1970).

so if you use eval to look for this difference you should have what you need.

sideview
SplunkTrust
SplunkTrust

Well more generally you want to see anywhere where there's a delta between _indexTime and _time If both sides respect DST there's no issue. If only one respects DST there'll be a persistent delta suddenly, and if they're slightly off there'll be a delta for some small slice of time and then it'll go away. And in each case which direction the delta goes depends on which side is observing DST and which is not.

0 Karma

araitz
Splunk Employee
Splunk Employee

This assumes that your Splunk servers' clocks are affected by DST 🙂

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...