Splunk Search

How to create a search that will trigger an alert when there are no events?

sravankaripe
Communicator

I want to trigger an alert if there are no events in the selected time range. please help me with sample search.

1 Solution

gcusello
SplunkTrust
SplunkTrust

Hi sravankaripe,
the best way to monitor if Splunk doesn't receive events for some fields value it's to build a lookup with all the expected field values and run a search like this:

| inputlookup mylookup.csv | eval count=0, myfield=lower(myfield) | append [ search mysearch | eval myfield=lower(myfield) | stats count by myfield ] | stats sum(count) AS total by myfield | where total=0

If you like, you can add a rangemap command and visualize the situation in a graphic panel.

Bye.
Giuseppe

View solution in original post

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi sravankaripe,
the best way to monitor if Splunk doesn't receive events for some fields value it's to build a lookup with all the expected field values and run a search like this:

| inputlookup mylookup.csv | eval count=0, myfield=lower(myfield) | append [ search mysearch | eval myfield=lower(myfield) | stats count by myfield ] | stats sum(count) AS total by myfield | where total=0

If you like, you can add a rangemap command and visualize the situation in a graphic panel.

Bye.
Giuseppe

0 Karma

skoelpin
SplunkTrust
SplunkTrust

This is called a negative alert since you want to alert on the presence of nothing being available..

Luckily, this is very easy to do in Splunk!

Define your index and sourcetype. Then save it as you would with any alert, then under "Alert Conditions" select Number of results and choose is equal to then put 0 in your field next to it.

You also need to set the time window you want the search to look in

Boom, you have a negative alert!

sravankaripe
Communicator

[alert1]
action.email = 1
action.email.inline = 1
action.email.reportServerEnabled = 0
action.email.sendresults = 1
action.email.to = mymail@mail.com
alert.expires = 7d
alert.suppress = 0
counttype = number of events
cron_schedule = *
dispatch.earliest_time = rt-1m
dispatch.latest_time = rt-30s
enableSched = 1
quantity = 0
relation = greater than
search = my search

i have this options because we are dealing with service-now incidents

0 Karma

skoelpin
SplunkTrust
SplunkTrust

You pull this from your savedsearches.conf?

If so then it would look something like this

[Logserver_Output Stopped Logging]
action.email = 1
action.email.include.trigger_time = 1
action.email.inline = 1
action.email.sendresults = 1
action.email.to = xxx@email.com
action.email.useNSSubject = 1
alert.suppress = 0
alert.track = 1
counttype = number of events
enableSched = 1
quantity = 1
relation = less than
request.ui_dispatch_app = search
request.ui_dispatch_view = search
search = index=endeca sourcetype=logserver_output
0 Karma

skoelpin
SplunkTrust
SplunkTrust

@sravankaripe , did this solve your problem? If so can you accept the answer and close it out?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...