Alerting

How to create an alert triggered by the presence of a specific syslog message, but absence of another within a five minute interval?

lagnone_splunk
Splunk Employee
Splunk Employee

Looking for assistance in crafting a scheduled search that sends a notification when I see a specific syslog message AND the absence of a specific second message in a given time period (say 5 minutes).

Example:

2014-05-10 11:30:00 NOTICE System restarted bla bla bla bla (This gets logged)
2014-05-10 11:30:15 INFO This is another log
2014-05-10 11:30:40 INFO This is a another log
2014-05-10 11:31:00 INFO All good dude ( I want to get notified when I don't get this)

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

You could do something like this:

base search with "Message One" OR "Message Two" | stats count(eval(match(_raw, "Message One"))) as ones count(eval(match(_raw, "Message Two"))) as twos

That'll give you two counts that you can use in your alert trigger condition, something like this:

where ones > 0 AND twos = 0

That'll trigger if message one is present but message two isn't.

Note, this doesn't do anything smart about the time between those messages or their order. Do you need to take care of that?

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to May Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars this month. This ...

They're back! Join the SplunkTrust and MVP at .conf24

With our highly anticipated annual conference, .conf, comes the fez-wearers you can trust! The SplunkTrust, as ...

Enterprise Security Content Update (ESCU) | New Releases

Last month, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...