In broad terms, I am searching for a certain event type and figuring out which state things were in for each event, where the state change is signified by other events.
For example, say I have a heartbeat event, and I have "became happy" and "became sad" events. I am trying to determine, at each heartbeat, whether it was happy or sad. I am having a really hard time figuring out how to pull this off.
The main avenue that I have pursued was to try and do a subsearch for the state change events with "latest=" the time of each heartbeat event, but "latest" can only be assigned a literal string. I.e., I've tried something like these two attempts, but they do not work:
event=heartbeat
| eval heartbeatTime=_time
| eval happy=1
| join type=left id [ search event=became.happy OR event=became.sad
| where _time<heartbeatTime
| eval happy=if(event=="became.happy", 1, 0)
| dedup id
]
or
event=heartbeat
| eval heartbeatTime=_time
| eval happy=1
| join type=left id [ search latest=heartbeatTime event=became.happy OR event=became.sad
| eval happy=if(event=="became.happy", 1, 0)
| dedup id
]
Does this produce a sample data set matching your question?
| stats count | eval count = 1 | eval event = "became.happy heartbeat heartbeat became.sad heartbeat became.happy became.sad heartbeat became.happy heartbeat" | makemv event | mvexpand event | accum count | eval _time = now()-count | fields - count
If so, append this to calculate a state
field:
... | eval state=case(match(event, "^became"), replace(event, "became.", "")) | filldown state
The producing search before that would be this:
event=heartbeat OR event=became.happy OR event=became.sad
Does this produce a sample data set matching your question?
| stats count | eval count = 1 | eval event = "became.happy heartbeat heartbeat became.sad heartbeat became.happy became.sad heartbeat became.happy heartbeat" | makemv event | mvexpand event | accum count | eval _time = now()-count | fields - count
If so, append this to calculate a state
field:
... | eval state=case(match(event, "^became"), replace(event, "became.", "")) | filldown state
The producing search before that would be this:
event=heartbeat OR event=became.happy OR event=became.sad
You can use "|sort count=0 fieldname" to eliminate 10000 limit.
"sort" restricts the number of events down to 10,000, so I did a "reverse"; it's easier and doesn't trim the events.
Looking at it again, you may need to sort by time before the filldown
to get the events after a state change affected by that very change rather than the events before.
I will try this out today; the filldown seems to be the missing piece; I can then filter out those state change events after the filldown.