- Splunk Answers
- :
- Using Splunk
- :
- Alerting
- :
- Why are email alerts not sent until Splunk restart...
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
We have had a problem over the weekend when one of our alerts did not trigger. I had to restart the services to get it all working again.
Does anyone had any idea why this might have happened?
It's possible it was related to changes we had made. It's the second time in a week we've needed to restart the services to have changes start working.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
In the end it appeared that the splunk server was skipping triggering as apparently there is a limit to 1 real time alert per CPU core.
We increased this and it mostly fixed the issues.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
In the end it appeared that the splunk server was skipping triggering as apparently there is a limit to 1 real time alert per CPU core.
We increased this and it mostly fixed the issues.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
What changes did you make and how (deployed/updated conf files/from UI)?
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I create a new lookup table, added the new fields to search and also to the email alert that went out.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I should say i tested it and it was working 11.55pm on friday. Then nothing for the rest of the weekend
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Did you check the scheduler logs for whether the alert search was run and if there were results that would trigger the alert?
index=_internal sourcetype=scheduler savedsearch_name="YourAlertName"
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks for this, yes it shows that it ran 1441 times on that day. Meaning it ran every minuet in the day so all working well..
Also if i run the search that the alert is built on the event shows up so i know the criteria was met.
This morning i sent a test alert no email
restarted services
sent another test alert and it worked.
Introducing the 2024 SplunkTrust!
Introducing the 2024 Splunk MVPs!
Splunk Custom Visualizations App End of Life
