Splunk Search

Correlating Different Events and Calculating Time Difference

dw385
Explorer

I’m trying to report on the time difference between two related events. Both events are collected from Windows event log and are in the same index. They will occur on the same host but at different times and IDs.
Example event data:

Time: 3/11/2016 9:00:45 PM
EventID: 1000
Source: Backup Agent Message:
Backup Started
Host: Server1
Time: 3/12/2016 1:09:45 AM

EventID: 1001
Source: Backup Agent
Message: Backup Completed
Host: Server1

The second event can occur any time after the first, maybe in a few minutes or a few hours. I’m thinking I need to find event 1(1000) first and then look for the first occurrence of event 2 afterwards. The index would contain similar events from many hosts around the same time.
I’ve tried various attempts with joins, transactions, and subsearches and searching the documentation for suggestions with no luck. I’m really not concerned with any of the data in the event other than the time (which I would eval to a start and end time). The goal is to be able to run a report over a time span to show the start/end times for each host.

This gets me close but doesn’t scale to multiple hosts/time ranges:

index="wineventlog" SourceName="Backup Agent" (EventCode="1000" OR EventCode="1001") 
| transaction maxspan=4h startswith=(EventCode="1000") endswith=(EventCode="1001") 
| table host,duration
Tags (1)
0 Karma

somesoni2
Revered Legend

How many times these events can happen from one host in a day?

0 Karma

dw385
Explorer

Usually only one occurrence but its possible to see multiple in a day as jobs are restarted.

0 Karma

lguinn2
Legend

Try this first:

 index="wineventlog" SourceName="Backup Agent" (EventCode="1000" OR EventCode="1001") 
 | transaction host maxspan=4h startswith=(EventCode="1000") endswith=(EventCode="1001") 
 | table host,duration

All I did was add host to the transaction command. This should help, but it still may not be a good solution, if you have a long time range.
You could also try this:

 index="wineventlog" SourceName="Backup Agent" (EventCode="1000" OR EventCode="1001") 
| sort host,_time
| eval starttime=if(EventCode="1000",_time,null())
| eval endtime=if(EventCode="1001",_time,null())
| streamstats current=t window=2 global=f latest(starttime) as start latest(endtime) as end by host
| where isnotnull(endtime)
| eval duration=end-start

This solution should work with somewhat larger timeframes. I am not sure how well it will work if you have multiple backups for a single host.

0 Karma
Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...