All Apps and Add-ons

Splunk Alert

gajananh999
Contributor

Dear All,

We are using splunk for monitoring purpose. We are pulling data from multiple application We want fire an alert suppose if i wont get the one statement in particular log file. I get a one event at every two hour so whenever i get that event i get a alert. But my requirement is different if i wont get this event in 2 hours then i have to get a alert . Can anybody help me how to get this done?

Thanks

Gajanan Hiroji

0 Karma
1 Solution

martin_mueller
SplunkTrust
SplunkTrust

You could have your alert trigger whenever there are zero results. For example, run this realtime with a two-hour window:

your search terms | stats count

Configure the alert's trigger condition to fire when count is zero. Make sure to also set throttling.

View solution in original post

renems
Communicator

Also, you could evaluate the search, by calculating a "gap". Then alert when your gap is too long.
This is how I did it:

| stats max(_time) As LatestTime by appserver | eval gap=(now()-LatestTime) | eval appserver="WebSphere" | sort num(gap) D | head 1 | rangemap field=gap low=0-300 default=severe

renems
Communicator

But Martin is right, since you want to alert after 2 hours, your search has to span at least 2 hours. I don't know what amount of data you have, but that could be an expensive search if it would span a lot of buckets.

0 Karma

renems
Communicator

with the "| eval gap=(now()-LatestTime)" you create a field that sets a value for the time that has elapsed since your event was recorded. In this example, the rangemap will turn to red if "gap" is greater then 300 secs.

0 Karma

gajananh999
Contributor

Thanks Renems. Can you elaborate this i am not able to get the | eval appserver="WebSphere" | sort num(gap) D | head 1 | rangemap field=gap low=0-300 default=severe

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Note, you need to use a time range longer than two hours for this to work... Long enough to be sure that the most recent event is still visible but older than two hours when running the alert.

martin_mueller
SplunkTrust
SplunkTrust

You could have your alert trigger whenever there are zero results. For example, run this realtime with a two-hour window:

your search terms | stats count

Configure the alert's trigger condition to fire when count is zero. Make sure to also set throttling.

gajananh999
Contributor

Yes if the event doesn't come in 2 hour then i have fire a alert.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Ah. Select cron schedule and enter this:

0 */2 * * *

That'll run at 00:00, 02:00, 04:00, etc.

If there's an event at 00:30 and one at 03:00 that's more than two hours... should your alert fire in this case? Scheduling this with a two-hour time range every two hours would not fire because there was an event in each two-hour time range despite the difference between the events being greater than two hours.

0 Karma

gajananh999
Contributor

Whenever i am scheduling alert the option for time range is 1) Every hour 2) every day kind of thing but there is no every two hour option where i have to do this conf files?

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

I don't think I understand that question, please elaborate on what you're asking.

0 Karma

gajananh999
Contributor

Thanks for your reply but when there is alert we can schedule for 2 hours in splunk.?

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...