Alerting

Supress saved search alert during holidays

t_splunk_d
Path Finder

I want to supress splunk alert during holidays.
I have a holidays.csv lookup.
For example: my search is like below:

index=abc sourcetype=something "look for events"

in the alert condition- if the record count is less than 3 then i trigger the alert.

I don't want to trigger this alert on a holiday when it runs ( the record count will be 0) it will fire a false alert.
and i want to retain the search and condition because during the business day i want it to fire if it returns less than 3.

Also I have another alert which fires has to suppressed holiday+1 day with the same condition as above.

Any suggestions?

Tags (2)
0 Karma

bhavikbhalodia
Path Finder

Hi,

You can configure your requirements in a query. You have to manage your query in such a way that when a search of alert runs holidays it should return 0 events. You can use addinfo and lookup commands.

If you are satisfied with a comment then please upvote it.

Thanks,
Bhavik

0 Karma

tom_frotscher
Builder

Hi,

how does your holidays.csv look like? Does it contain timestamps, ranges of dates?

There might be many ways to the right answer. You can for example take atheeval command and create a field based on _time that only has year.month.day left. If your holidays.csv file also is just a list of such year.month.day entries for every day that you consider a vacation day, you can just use it as a lookup and filter out every matching event.

Greetings,

Tom

0 Karma

t_splunk_d
Path Finder

Thanks, but my question is not to filter out. I am able to match the holiday, but not able to suppress alert on holidays.

Format of holidays.csv
Date. description
12/25/2018 Christmas

0 Karma

tom_frotscher
Builder

Ok, got it now.

I don't think there is a out of the box solution for this.

I think you can append the csv with append and then check if any entry in the holiday csv is today:

append [|inputlookup holiday.csv | eval time = strftime(now(),"%d/%m/%Y") | where time == Date]

If your search returns something, the append should be empty, because it is not a holiday. If your search returns nothing, the append will result in an entry, because it is a holiday.

Only Problem is, you get only one event instead of 3, so you would need 3 times the entry in your holidays.csv. Or you reconfigure your trigger condition. You should be able to use a custom trigger condition where you say trigger if count < 3 AND description = null. But therefore you might need to add a description field to the results of your search.

Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...