Splunk Search

How to receive an alert when a field value is missed?

nagarjuna280
Communicator

index="main" | stats count by sourcetype | search count>40000, I will get 10 sourcetypes, If any source type doesn't reach condition then I should get alert as "sourcetype name missed", I should get all source type names

Tags (3)
0 Karma
1 Solution

somesoni2
Revered Legend

You could place those 10 sourcetype names in a lookup (scalable solution) and then append that to your current search. You can also, use a subsearch with directly providing the sourcetype name. Something like this

with lookup

Assuming you create a lookup called sourcetype_to_track.csv with single field name 'sourcetype' and it contains all the sourcetype that you want to check.

index="main" | stats count by sourcetype | search count>40000 | append [| inputlookup sourcetype_to_track.csv | table sourcetype | eval count=0] | stats max(count) as count by sourcetype | where count=0

directly providing sourcetype names in the subsearch.

index="main" | stats count by sourcetype | search count>40000 | append [| gentimes start=-1 | eval sourcetype="sourcetype_to_track1,sourcetype_to_track2..list all sourcetypes here separated by comma" | table sourcetype | makemv sourcetype delim="," | mvexpand sourcetype| eval count=0] | stats max(count) as count by sourcetype | where count=0

In both case setup your alert to get fired when 'number of results in greater than 0'. This should give you list all sourcetype where count was not > 40000.

View solution in original post

somesoni2
Revered Legend

You could place those 10 sourcetype names in a lookup (scalable solution) and then append that to your current search. You can also, use a subsearch with directly providing the sourcetype name. Something like this

with lookup

Assuming you create a lookup called sourcetype_to_track.csv with single field name 'sourcetype' and it contains all the sourcetype that you want to check.

index="main" | stats count by sourcetype | search count>40000 | append [| inputlookup sourcetype_to_track.csv | table sourcetype | eval count=0] | stats max(count) as count by sourcetype | where count=0

directly providing sourcetype names in the subsearch.

index="main" | stats count by sourcetype | search count>40000 | append [| gentimes start=-1 | eval sourcetype="sourcetype_to_track1,sourcetype_to_track2..list all sourcetypes here separated by comma" | table sourcetype | makemv sourcetype delim="," | mvexpand sourcetype| eval count=0] | stats max(count) as count by sourcetype | where count=0

In both case setup your alert to get fired when 'number of results in greater than 0'. This should give you list all sourcetype where count was not > 40000.

Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...