Splunk Search

ideas for reliably bracketing timerange around discrete 'snapshots'

sideview
SplunkTrust
SplunkTrust

Say that you have a huge volume of events, and they come in big batches. Each batch is a discrete unit, and mixing information from the most recent batch with the previous batch is unacceptable.

more givens:

  1. the events within a particular batch are spread out over a few minutes.
  2. we do have control over the data so we could write a particular event at the start and at the end of the batch if necessary. We could even create a start/end event that had a different source or sourcetype.

Given all this, Is there a good clean way to construct a custom search or a custom view that will be sure to operate only on the events of the most recent batch?

Tags (1)
1 Solution

Stephen_Sorkin
Splunk Employee
Splunk Employee

Your best bet is to use the "head" command which can take a predicate instead of an absolute count.

For example, the following search only takes (all of) the events from the most recent second from index=_internal:

index=_internal | streamstats dc(_time) as dc_time | head dc_time==1

View solution in original post

Stephen_Sorkin
Splunk Employee
Splunk Employee

Your best bet is to use the "head" command which can take a predicate instead of an absolute count.

For example, the following search only takes (all of) the events from the most recent second from index=_internal:

index=_internal | streamstats dc(_time) as dc_time | head dc_time==1

Stephen_Sorkin
Splunk Employee
Splunk Employee

This ability of head has been around since 4.1, I believe.

0 Karma

sideview
SplunkTrust
SplunkTrust

transaction wont work here because the set of events needs to be sliced and diced up a number of ways by a lot of different 'stats foo(bar) by baz, bat' searches, and transaction is going to put me in multivalue hell.

0 Karma

Brian_Osburn
Builder

Wouldn't this be a good use of a transaction command? especially if you've got a well defined start and stop?

0 Karma

sideview
SplunkTrust
SplunkTrust

The events here are not in a single second, but this offers a tool that seems to open up a number of other ideas. Is that ability of the head command new in 4.2? It seems like I could use eval and streamstats to keep track of when I see the 'start' event and 'end' event, and then use head to terminate once I get back to the correct head event. Is that what you would do?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...