I have 36 servers that forward event sources with 2 distinct values. I need to compare the number of system names (from previous 24 hours) * 2 and compare that result to a dedup of system_name, and event_source (from past 60 minutes). If the results don't match, I need an alert. One more problem is, the number of system names is subject to change.
index=myIndex sourcetype=mySourcetype
[ search index=myIndex sourcetype=mySourcetype earliest=-1d
| dedup system_name
| stats count(eval(system_name*2)) as SysNmCount ]
[ search index=myIndex sourcetype=mySourcetype earliest=-60m
| dedup event_source, system_name
| stats count as EvntCount ]
| stats count(eval(SysNmCount/EvntCount)) as FinalCount
This search returns 0. I was thinking I could use a
| Where
clause to alert when the FinalCount doesn't equal 1 when I get the return I need. I would like to include the
dedup event_source, system_name
in the alert email so the system operators can start looking for the missing combination(s) problem.
The search that works right now has the value of 72, but it changes from time to time.
index=myIndex sourcetype=mySourcetype
| dedup event_source, system_name
| eventstats count as total
| where (total < 72 OR total > 72)
| fields system_name event_source
Any suggestion would be appreciated.
See final comment for the solution that works.
Researching got me a little further. Since I have 2 subsearches, I use "set union". Now I need a table that is created ONLY when SysNmCount and EvntCount are different.
Any suggestions?
| set union
[search index=myIndex sourcetype=mySourcetype earliest=-60m@m
| dedup system_name event_source
| stats count as EvntCount]
[search index=myIndex sourcetype=mySource earliest=-24h
| dedup system_name
| stats count as total
| eval SysNmCount = total * 2]
| table SysNmCount EvntCount
I finally figured it out. Instead of "diff" or "union" or "makeresults" there is a simple solution. Sorry, I'm still new to SPL.
index=myIndex sourcetype=mySourcetype earliest=-24h NOT
[search index=myIndex sourcetype=mySourcetype earliest=-1h
| dedup system_name event_source
| fields system_name event_source
| table system_name, event_source]
| dedup system_name event_source
| fields system_name event_source
| table system_name, event_source
| eventstats count as total
| where total > 0
Setting this to run every hour (at 8 minutes past for resource issues) as an alert returns the event sets that are there for 24 hours but not the previous hour.