Knowledge Management

is there a way to extract raw event data from a summary index?

sahanapranesh
New Member

Please, could you let me know if there is a way to extract the raw data of an event from the summary index report?
I have a summary index report like:

index=main earliest=-1445m@m latest=-5m@m "Operation1" OR "Operation2"
| fields _time field1 field2
| stats min(_time) max(_time) by field1 field2

On top of which we fire a search:

index=summary report = si_SummaryIndexReport | stats min(_time) as earliest max(_time) as latest by field1 field2 | where latest>earliest | eval duration=(latest-earliest)/86400 | table field1 field2 earliest latest duration

Basically I want to calculate the time between the first occurrance of Operation1 and the last occurrance of Operation3.
The behavior should be similar to transaction command where we give the "startsWith" and "endsWith" attributes.
Due to performance issues I cannot use the transaction command.

This search behaves in the following way
Scenario 1: Logs for Success case
15/08/2015: Operation1 for field1=112233 –min(_time) will show 15/08/2015
16/08/2015: Operation2 for field1=112233
20/08/2015: Operation1 for field1=112233
26/08/2015: Operation3 for field1=112233 –max(_time) will show 26/08/15

This is the expected behavior and the search works fine for scenario 1.

Scenario 2: Logs for Failure case
15/08/2015: Operation1 for field1=112233 –min(_time) will show 15/08/2015
16/08/2015: Operation2 for field1=112233
20/08/2015: Operation1 for field1=112233 –max(_time) will show 20/08/2015

In this case there have been 2 events for Operation1 alone, hence it should not even be considered for the final table.
So, please let me know if there’s a way of extracting event data or repopulating an already existing summary index with a newly extracted field, say status, where status could be “Operation1” “Operation2” or “Operation3”
I want min(_time) to ALWAYS be a “Operation1” event and a max(_time) to ALWAYS be a “Operation3” event.

Thanks in advance!
Sahana

0 Karma
1 Solution

somesoni2
SplunkTrust
SplunkTrust

Try this for your summary index search (assuming you've a field called Operation already extracted)

index=main earliest=-1445m@m latest=-5m@m "Operation1" OR "Operation2" OR "Operation2"
| fields _time field1 field2
| stats min(_time) max(_time) values(Operation) as Operations by field1 field2 | where isnotnull(mvfilter(match(Operations,"Operation3"))) | fields - Operations

This way, if your grouping of field1 and field2 will be discarded if it doesn't include Operation3

View solution in original post

0 Karma

somesoni2
SplunkTrust
SplunkTrust

Try this for your summary index search (assuming you've a field called Operation already extracted)

index=main earliest=-1445m@m latest=-5m@m "Operation1" OR "Operation2" OR "Operation2"
| fields _time field1 field2
| stats min(_time) max(_time) values(Operation) as Operations by field1 field2 | where isnotnull(mvfilter(match(Operations,"Operation3"))) | fields - Operations

This way, if your grouping of field1 and field2 will be discarded if it doesn't include Operation3

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...