Splunk Search

Keeping track of a state for each event by looking for previous state changes

Raistlan
Explorer

In broad terms, I am searching for a certain event type and figuring out which state things were in for each event, where the state change is signified by other events.

For example, say I have a heartbeat event, and I have "became happy" and "became sad" events. I am trying to determine, at each heartbeat, whether it was happy or sad. I am having a really hard time figuring out how to pull this off.

The main avenue that I have pursued was to try and do a subsearch for the state change events with "latest=" the time of each heartbeat event, but "latest" can only be assigned a literal string. I.e., I've tried something like these two attempts, but they do not work:

event=heartbeat
| eval heartbeatTime=_time
| eval happy=1
| join type=left id [ search event=became.happy OR event=became.sad
                    | where _time<heartbeatTime
                    | eval happy=if(event=="became.happy", 1, 0)
                    | dedup id
                    ]

or

event=heartbeat
| eval heartbeatTime=_time
| eval happy=1
| join type=left id [ search latest=heartbeatTime event=became.happy OR event=became.sad
                    | eval happy=if(event=="became.happy", 1, 0)
                    | dedup id
                    ]
Tags (2)
0 Karma
1 Solution

martin_mueller
SplunkTrust
SplunkTrust

Does this produce a sample data set matching your question?

| stats count | eval count = 1 | eval event = "became.happy heartbeat heartbeat became.sad heartbeat became.happy became.sad heartbeat became.happy heartbeat" | makemv event | mvexpand event | accum count | eval _time = now()-count | fields - count

If so, append this to calculate a state field:

... | eval state=case(match(event, "^became"), replace(event, "became.", "")) | filldown state

The producing search before that would be this:

event=heartbeat OR event=became.happy OR event=became.sad

View solution in original post

martin_mueller
SplunkTrust
SplunkTrust

Does this produce a sample data set matching your question?

| stats count | eval count = 1 | eval event = "became.happy heartbeat heartbeat became.sad heartbeat became.happy became.sad heartbeat became.happy heartbeat" | makemv event | mvexpand event | accum count | eval _time = now()-count | fields - count

If so, append this to calculate a state field:

... | eval state=case(match(event, "^became"), replace(event, "became.", "")) | filldown state

The producing search before that would be this:

event=heartbeat OR event=became.happy OR event=became.sad

somesoni2
SplunkTrust
SplunkTrust

You can use "|sort count=0 fieldname" to eliminate 10000 limit.

0 Karma

Raistlan
Explorer

"sort" restricts the number of events down to 10,000, so I did a "reverse"; it's easier and doesn't trim the events.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Looking at it again, you may need to sort by time before the filldown to get the events after a state change affected by that very change rather than the events before.

0 Karma

Raistlan
Explorer

I will try this out today; the filldown seems to be the missing piece; I can then filter out those state change events after the filldown.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...