Splunk Search

I need an alert that triggers when it can't find a certain message in the logs within the last hour

rgarbac1
New Member

It always brings up no results. Here is my query:

index=abc  host = "123" OR host = "456" OR host = "789" OR host = "012" fullload = "]I:  Task is running" | stats count by source, 
fullload | where count < 1 

It is looking for this message: "]I: Task is running"
can you please help

Tags (3)
0 Karma

mayurr98
Super Champion

hey you can try something like this
when there is "NO result found" processCount field will get the value of 0

 index=abc  host = "123" OR host = "456" OR host = "789" OR host = "012" fullload = "]I:  Task is running" | stats count by source, fullload   |  appendpipe [ stats count | eval processCount=0 | where count==0 | fields - count ] | search processCount=0

Run this for last 1 hour and schedule an alert to run every hour for last 1 hour .
Trigger alert when select custom and write condition processCount=0

let me know if this helps!

0 Karma

rgarbac1
New Member

Thanks I will try it.

0 Karma

rgarbac1
New Member

I need the search to bring up all occurrences of a count of 0 messages by source. There are 250 sources.

0 Karma

micahkemp
Champion

If you're going to search for "what's not there", you have to include in your search some default values of what should be there, so that you aren't just missing rows for the missing data. This run anywhere search shows how you can do this:

index=_internal sourcetype=mongod OR sourcetype=splunkd OR sourcetype=splunkd_conf OR sourcetype=madeup
| stats count BY sourcetype
| append [| makeresults | eval sourcetype="mongod,splunkd,splunkd_conf,madeup" | makemv delim="," sourcetype | table sourcetype | mvexpand sourcetype | eval count=0]
| stats sum(count) AS count BY sourcetype
| search count=0

The append line could use inputlookup to fetch the list of expected sourcetypes, or you could hardcode them into the search as shown here. It creates a row for each expected sourcetype with count=0, then performs another stats afterwards to sum them so you can see which events created via append didn't have any additional count from before.

0 Karma

rgarbac1
New Member

Thanks I will try it.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...