Dear All,
We are using splunk for monitoring purpose. We are pulling data from multiple application We want fire an alert suppose if i wont get the one statement in particular log file. I get a one event at every two hour so whenever i get that event i get a alert. But my requirement is different if i wont get this event in 2 hours then i have to get a alert . Can anybody help me how to get this done?
Thanks
Gajanan Hiroji
You could have your alert trigger whenever there are zero results. For example, run this realtime with a two-hour window:
your search terms | stats count
Configure the alert's trigger condition to fire when count is zero. Make sure to also set throttling.
Also, you could evaluate the search, by calculating a "gap". Then alert when your gap is too long.
This is how I did it:
| stats max(_time) As LatestTime by appserver | eval gap=(now()-LatestTime) | eval appserver="WebSphere" | sort num(gap) D | head 1 | rangemap field=gap low=0-300 default=severe
But Martin is right, since you want to alert after 2 hours, your search has to span at least 2 hours. I don't know what amount of data you have, but that could be an expensive search if it would span a lot of buckets.
with the "| eval gap=(now()-LatestTime)" you create a field that sets a value for the time that has elapsed since your event was recorded. In this example, the rangemap will turn to red if "gap" is greater then 300 secs.
Thanks Renems. Can you elaborate this i am not able to get the | eval appserver="WebSphere" | sort num(gap) D | head 1 | rangemap field=gap low=0-300 default=severe
Note, you need to use a time range longer than two hours for this to work... Long enough to be sure that the most recent event is still visible but older than two hours when running the alert.
You could have your alert trigger whenever there are zero results. For example, run this realtime with a two-hour window:
your search terms | stats count
Configure the alert's trigger condition to fire when count is zero. Make sure to also set throttling.
Yes if the event doesn't come in 2 hour then i have fire a alert.
Ah. Select cron schedule and enter this:
0 */2 * * *
That'll run at 00:00, 02:00, 04:00, etc.
If there's an event at 00:30 and one at 03:00 that's more than two hours... should your alert fire in this case? Scheduling this with a two-hour time range every two hours would not fire because there was an event in each two-hour time range despite the difference between the events being greater than two hours.
Whenever i am scheduling alert the option for time range is 1) Every hour 2) every day kind of thing but there is no every two hour option where i have to do this conf files?
I don't think I understand that question, please elaborate on what you're asking.
Thanks for your reply but when there is alert we can schedule for 2 hours in splunk.?