Alerting

How do I make my alert artifacts stick around longer?

MonkeyK
Builder

I need search results involved in alerts to be available for a longer period of time than they are now (currently, nobody on my team seems to get to the alert in time to see the alerting event).

I know that this question was asked before:

https://answers.splunk.com/answers/440040/saving-alert-artifacts-for-longer-periods-of-time.html

However, I cannot make that solution work. The solution is supposed to be that I run the alter search and set its job settings to have a longer lifetime. When I do that the setting reverts to 10m the next time I open the alert search up.

in my saved search listing, I do notice that there is an advanced edit with 600+ values, including 30 ttl value. Is one of those values the right thing to set?

Tags (1)
0 Karma

somesoni2
Revered Legend

You should setting attribute dispatch.ttl

dispatch.ttl = <integer>[p]
* Indicates the time to live (in seconds) for the artifacts of the scheduled
  search, if no actions are triggered.
* If the integer is followed by the letter 'p' Splunk interprets the ttl as a
  multiple of the scheduled search's execution period (e.g. if the search is
  scheduled to run hourly and ttl is set to 2p the ttl of the artifacts will be
  set to 2 hours).
* If an action is triggered Splunk changes the ttl to that action's ttl. If
  multiple actions are triggered, Splunk applies the largest action ttl to the
  artifacts. To set the action's ttl, refer to alert_actions.conf.spec.
* For more info on search's ttl please see limits.conf.spec [search] ttl
* Defaults to 2p (that is, 2 x the period of the scheduled search).
0 Karma

MonkeyK
Builder

Well then, something must be wrong.
in advanced edit, I had found
"dispatch.ttl" and set its value to 259200, (3 days). The search jobs seem as if they stay for 3 days: I can see create and expire dates like this:
Nov 20, 2017 9:20:01 PM Nov 23, 2017 9:20:04 PM

And yet I was alerted today at 12:20, the alert link (from the email) says "Page not found!" and the alert page says "There are no fired events for this alert." While job listing has the alert search job from 12:20 in which I can see the triggering event. So its as if the alert artifact has a shorter life than the alert serach.

0 Karma
Get Updates on the Splunk Community!

Detecting Remote Code Executions With the Splunk Threat Research Team

REGISTER NOWRemote code execution (RCE) vulnerabilities pose a significant risk to organizations. If ...

Observability | Use Synthetic Monitoring for Website Metadata Verification

If you are on Splunk Observability Cloud, you may already have Synthetic Monitoringin your observability ...

More Ways To Control Your Costs With Archived Metrics | Register for Tech Talk

Tuesday, May 14, 2024  |  11AM PT / 2PM ET Register to Attend Join us for this Tech Talk and learn how to ...