Alerting

Alert on |search field=0 returns too many rows?

bmgilmore
Path Finder

I've got a scheduled search that calculates the variability of a numeric field over time that should always be moving. If I pipe a "search Variability=0" at the end, and run in search view, the search runs, shows a lot of rows at first as it calculates back through time, and then shows the correct number of rows.

Oddly, when I set this search to alarm, it sends many alarms and the CSV search results attached show many (almost all) of the available rows, each with a Variability of 0. Returning to the triggered job returns 0 (or whatever 1 or 2) rows as I would expect. Especially strange is that each time the alert triggers, it is a new number of false positive rows.

It's almost as if the search is not waiting to complete before triggering the alert? Any ideas here? Thanks in advance!

Tags (1)
0 Karma

jonuwz
Influencer

You probably need a custom condition in your alert

i.e.

main search

... | stats count(eval(Variability==0)) as not_variable

then a custom condition where

search not_variable > 0
0 Karma

MHibbin
Influencer

attached? ... you will need to paste it in the question as code (using the button "101010", or by starting each line of the code with 4 spaces, or by inclosing in backticks).

0 Karma
Get Updates on the Splunk Community!

Get Your Exclusive Splunk Certified Cybersecurity Defense Engineer at Splunk .conf24 ...

We’re excited to announce a new Splunk certification exam being released at .conf24! If you’re headed to Vegas ...

Share Your Ideas & Meet the Lantern team at .Conf! Plus All of This Month’s New ...

Splunk Lantern is Splunk’s customer success center that provides advice from Splunk experts on valuable data ...

Combine Multiline Logs into a Single Event with SOCK: a Step-by-Step Guide for ...

Combine multiline logs into a single event with SOCK - a step-by-step guide for newbies Olga Malita The ...