Alert Triggering only once even if set to 'Per Result'

[ashutoshab]

I have created a scheduled alert that looks for results over a time period and if there are events, it has to send an email for every result. This email alert creates a ticket in our ticketing portal.
Incase if there are 10 results in that time period, Splunk should send 10 emails. But instead Splunk triggers only once and send all the results in one single email. This is weird and I am trying to find solution to it.

Below is my Alert Configuration:

Alert Type: Scheduled Run on Cron Schedule
Time Range: last 15 minutes.
Cron Expression: 0,15,30,45 * * * *
Trigger alert when: Number of results > 0
Trigger: For each Result
Throttle: Unchecked.

When the alert triggers, it generates only one alert with the first result and does not trigger anything for the rest. I want to know what I am missing. I see there are 10 results but only one alert.

[Sujithkumarkb]

@ashutoshab ,seems like your throttle alert is on , which will accumulate all the events every 15mins and react at once.
You can try scheduling the alert in real time with throttle disabled.

Else you can try something like below

The other way round can be to add a counter or a variable condition in your alert query like below.
example : | stats phone by state " into the search and create a custom alert trigger such as " eval count = if(search state =received,1,0) |search count =1 (in your case to trigger every event)".

Hope this helps.

[snigdhasaxena]

Hi @ashutoshab,

Is it possible to make your query generic and share ?

[anil15694]

Hey @ashutoshab,

Here you are creating schedule alert with time range 15 min so alert is getting run every 15 min. If you want your alerts should run and send email on every event so you should create real time alerts. Real - time alerts will be useful to monitor events or event patterns as they happen.

You can use this splunk documentation as reference to create real-time alerts.

https://docs.splunk.com/Documentation/Splunk/7.2.6/Alert/DefineRealTimeAlerts

[marcoscala]

I downvoted this post because not pertinent: if you have a scheduled alert it should fire more actions according to the number or results.

[marcoscala]

I downvoted this post because not pertinent: if you have a scheduled alert it should fire more actions according to the number or results.

[ashutoshab]

Yes, I know that, but for similar scheduled search, for some other query, I receive alert for every event. I mean, it has a different query but similar schedule of 15 mins. If there are 10 events, I receive 10 emails.

Here, it send only 1 email for everything.

[horsefez]

Hey @ashutoshab ,

which Splunk version are you using?
Also can you post the specific stanza from the savedsearches.conf

[ashutoshab]

I am using Splunk Enterprise 7.2.4

Below is my Stanza

[STANZA NAME]
action.email = 1
action.email.include.results_link = 0
action.email.include.view_link = 0
action.email.mailserver = localhost
action.email.message.alert = {"RANDOM TEXT
}}
action.email.priority = 1
action.email.subject = <RANDOM TEXT>
action.email.to = RANDOM EMAIL ADDRESS
alert.digest_mode = 0
alert.expires = 1h
alert.severity = 4
alert.suppress = 0
alert.track = 1
counttype = number of events
cron_schedule = 0,15,30,45 * * * *
description = RANDOM DESCRIPTION
dispatch.earliest_time = -15m
dispatch.latest_time = now
display.events.type = raw
display.general.type = statistics
display.page.search.mode = verbose
display.page.search.tab = statistics
enableSched = 1
quantity = 0
relation = greater than
request.ui_dispatch_app = search
request.ui_dispatch_view = search
search = <SEARCH STRING>

[ashutoshab]

Waiting for an answer. I feel this might be a bug.

[adonio]

what is your search looks like?

[ashutoshab]

index=<someIndexName> sourcetype="<someSourceType>" <SomeField>=* | table eventType, sender, headerFrom, recipient{}, toAddresses{}, subject, imposterScore, GUID, messageTime, phishScore, spamScore, quarantineFolder, senderIP, messageID, threatsInfoMap{}.classification, threatsInfoMap{}.threatUrl, threatsInfoMap{}.threatID, threatsInfoMap{}.campaignID, threatsInfoMap{}.threat, threatsInfoMap{}.threatStatus, threatsInfoMap{}.threatTime, threatsInfoMap{}.threatType