Why is the custom trigger not firing for this alert?

[ihiesbkalai]

My search is:

index=soma   source="alarms.txt" StatusID=*
|eval Alarm=if(StatusID=0,"Critical","No")
| table  DateTime EventCode StatusID Message Alarm

The alert runs every minute (cron * * * * *)
and my custom trigger is
search Alarm="No"
There are no fired events for this alert
If I run the search |where Alarm="No" there are 94 events matching that

What am I doing wrong?

[micahkemp]

When you say cron *, what is your exact cron expression? I don't believe a single * is a valid expression, so I want to make sure the issue isn't simply that your search isn't running when you think it should be.

[ihiesbkalai]

the cron job value is * * * * *
And that's not the issue any longer. Triggers are fired but no emails is generated

[elliotproebstel]

What is the time window being monitored for this search that's running every minute? In my experience, if you're running a search every minute (and presumably looking at data logged in the previous minute), the delays of ingestion may prevent you from seeing anything at all.

[ihiesbkalai]

Last 7 days. this is a static file i uploaded to the index=soma

[elliotproebstel]

Got it. So how about changing the approach. I'd suggest changing your search to this:

index=soma source="alarms.txt" StatusID=* 
| eval Alarm=if(StatusID=0,"Critical","No") 
| table DateTime EventCode StatusID Message Alarm
| search Alarm="No"

And set the trigger to be Number of Results, with is greater than set to 0. Will that achieve what you're trying to do?

[ihiesbkalai]

will i get an alert of each instance (that's the desired affect)

[ihiesbkalai]

I made the change and I still get There are no fired events for this alert

[elliotproebstel]

What type of license do you have? Is there any chance your evaluation has expired? If you're in "Free" status, alerts won't fire.

If that's not the issue, check your Alert configuration. Ensure that under "Add Actions", you've selected "Add to Triggered Alerts".

And if you want one alert for each result, look at the "Trigger" line under "Trigger Conditions". There are two options: "Once" and "For each result". If you want one alert per result, select "For each result".

[ihiesbkalai]

I added Added Triggered Alerts to the trigger action and got a trigger but not an email

[elliotproebstel]

Then we're on the right track! Under "Trigger Actions", select "Add Actions" and choose "Send email". You'll need to enter your email address in the To field and then save it again.

[ihiesbkalai]

I setup the email action but no emails

[elliotproebstel]

Have you used this Splunk instance to send any other emails? If not, you'll need to follow the steps on this guide to configure communication between Splunk and your mail server:
http://docs.splunk.com/Documentation/Splunk/7.0.2/Alert/Emailnotification

[ihiesbkalai]

Splunk Enterprise Sales Trial 10,240 MB Mar 14, 2018 5:37:43 PM valid