Alerting

How to run different alerts/report based on output of some some Alert ?

tarunmalhotra79
Engager

Dear Splunker,

I am trying to implement end to end monitoring where searches have dependecy on multiple lookups and those lookups are dervied from different searches running internally.The idea is to diagnose the situation thouroughly and as early as possible without running all the searches/alerts all the time because there are atleast 60/70 Odd searches in production.
Please look at the below example for more clarity.

search 1
|inputlookup Lookup1
|rename sys_id AS app_sys_id
|lookup Lookup2 parent AS app_sys_id OUTPUTNEW child AS server_sys_id
|mvexpand server_sys_id
|join server_sys_id [inputlookup Lookup3 | rename sys_id AS server_sys_id]
|fields host, server_fqdn, server_status, server_support_group, server_type, server_sox, server_sas, server_admin, server_location, server_environment
|outputlookup Lookup1
Also, as we can see here Lookup1 is dependent on 2 lookups internally (Lookup2,Lookup3)

Search 2
sourcetype=someother_source2|outputlookup Lookup2
here, this sourcetype will eventually create lookup2

Search 3
sourcetype=someother_source3|outputlookup Lookup3
here, this sourcetype will eventually create lookup3

Basically, It's just an example in production I have more than 30 searches in such a manner,I do not want to create 30 alert and running all unnecessary at 30 Odd timings.

I was thinking if i can create something where I will create an alert which will check inside search 1, if the result count is zero or not,if it is zero I should be alerted and then only it should also internally run a report (search 2) to check there if a data is missing or not. If not then move to search 3 to check further, if something is missing then we need to be alerted that data is missing with search 2 or search 3 .It will help me in diagnosing the situation early and also not all the time unnecessary alerts are running on my search head.

Any ideas would be appreciated, I am okay using alerts,report,dashboards,scripts etc.

Again, thanks in advance.

Labels (1)
0 Karma

tarunmalhotra79
Engager

Did anyone get a chance to look into my request, any idea would be highly appreciated?

0 Karma

to4kawa
Ultra Champion
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...