Alerting

alert in splunk

loveforsplunk
Explorer

start time = -2h@h Finish Time = now.

Can I use this in an alert in Splunk.. which will run my search every minute for the last 2 hours and will throw an alert if gets any result.

Tags (1)
0 Karma
1 Solution

dineshraj9
Builder

Try running the search once and check how much time it takes. If you have huge volume of data being searched then ensure that there are not 2 concurrent execution. Setting up every minute for 2hr window would work fine, but if you are looking for a keyword in the raw events, it is advisable that the search window and frequency remain same.(search -2m@m to -1m@m every minute).

Please check for timezone issue with the logs and fix them if any.

View solution in original post

0 Karma

dineshraj9
Builder

Try running the search once and check how much time it takes. If you have huge volume of data being searched then ensure that there are not 2 concurrent execution. Setting up every minute for 2hr window would work fine, but if you are looking for a keyword in the raw events, it is advisable that the search window and frequency remain same.(search -2m@m to -1m@m every minute).

Please check for timezone issue with the logs and fix them if any.

0 Karma

loveforsplunk
Explorer

I have one question here.. I am getting the alert when I kept the search as -2h@h to now but it seems it is taking from the beginning of hour.

For ex: Suppose the search returns result at 10:36:18 AM and I get an alert at 12:36:18 PM . But I think it is searching from the beginning of hour that is 10 AM so I keep on getting continuous alert. Can You please suggest anything .

0 Karma

somesoni2
Revered Legend

When the search frequency and time range are different (search runs every minute but looks back 2 hrs every time) there are events which will be overlapping in multiple alert execution, hence duplicate alerts. Best option is to keep them in sync, as suggested by @dineshraj9. Also, unless it's really critical, setup a less frequent alert like every 10 or 15 mins instead of every minute. Also, in time range, allow some additional gap to account for indexing latency.

E.g. Search time range earliest=-12m@m latest=-2m@m with cron as 2/10 * * * *

loveforsplunk
Explorer

oh .. I get it now .. Thanks a lot for instant reply. And also thanks to Dinesh .. I did not get what he wanted to express 😉 Sorry Dinesh but Thanks to both of you. 🙂 I am very new to Splunk so asking silly questions . Hope you didn't mind 🙂

Get Updates on the Splunk Community!

Introducing Splunk Enterprise 9.2

WATCH HERE! Watch this Tech Talk to learn about the latest features and enhancements shipped in the new Splunk ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...