Splunk Search

How to edit my search to prevent getting multiple alerts?

sonusngh68
New Member

Created a search to monitor members added/removed from a group. It's working in search, but in the alert email for deletion of one member from the group, we're getting thousands of alerts. Please see the search and correct if I did anything wrong.

index=wineventlog sourcetype=WinEventLog:Security (EventCode=4728 OR EventCode=4729) Group_Name=SGG_Emergency_Database_Access Group_Domain=HSUSERS
| eval Date=strftime(_time, "%Y/%m/%d %H:%M:%S")
|rex "Member:\s+\w+\s\w+:.*\\\(?.*)"
| rex "Account\sName:\s+(?.*)"
| stats count by Date, TargetAccount, SourceAccount,ComputerName,Group_Name,Group_Domain,Keywords,name | sort - Date
| rename name as "Message"
| rename SourceAccount as "Administrator Account" 
| rename TargetAccount as "Target Account"
0 Karma

varad_joshi
Communicator

Can you please try this -

Go to alert - Alert Mode - Select Once per search. -- This should restrict it to one alert per search.

THEN

The option below is Throttling.

Check that box for ' After triggering the alert, don't trigger it again for'
Mention the time after which next alert is about to run.

As @Somesoni2 mentioned, do not keep it for real time. Keep it to run every 5 minutes.

Change throttling value according to your run time and see if this helps.

0 Karma

somesoni2
SplunkTrust
SplunkTrust

How many rows this search returns? Could you provide other information about the alert like time range, schedule, alert condition, throttling and alert type (per-result OR once-per-result)?

0 Karma

sonusngh68
New Member

It returns one row and have selected per result. But don't know why it is sending multiple email alerts for one member added or removed.

0 Karma

somesoni2
SplunkTrust
SplunkTrust

What about the schedule and time range? If you've overlapping timerange (e.g. running every 5 min and time range is last 30 mins, there is 25 mins overlapping between each alert execution), it'll cause repeated alert.

0 Karma

sonusngh68
New Member

Selected All Time(Real Time). What time range should I select ?

0 Karma

somesoni2
SplunkTrust
SplunkTrust

That's not good. I generally avoid real-time search, specially scheduled real-time search. If you're OK with 6 min delay in getting the result, they use this

Start time:  -6m@m 
End time: -1m@m
Cron Schedule: 1-59/5 * * * *
0 Karma

sonusngh68
New Member

From time picker what time range do I need to select?

0 Karma

sonusngh68
New Member

While scheduling this search to alert

0 Karma

somesoni2
SplunkTrust
SplunkTrust

Go to advanced section in the time-range picker, and use earliest as -6m@m and latest as -1m@m.

0 Karma

sonusngh68
New Member

As you instructed to change in alert Start time: -6m@m
End time: -1m@m
Cron Schedule: 1-59/5 * * * *

I did it and checked in time picker and picker is showing custome and same time has already updated. But not getting the result.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...