Hi, for a given index and sourcetype I have a field called host which can be one of 4 values
There normally several log-lines for each host every minute
I would like an alert if one of the hosts stops logging for say 5 minutes
I have
index=myindex sourcetype=mysourcetype earliest=-5m | stats count by host
and the alert to trigger if the value is 0 - but if one of the hosts is no in the logs - you get
host count
host-a 128
host-b 143
host-d 122
and host-c is missing rather than 0 so nothing triggers 😞
any ideas other than 4 separate alerts?
Hi stephenmoorhouse,
I suggest to you to insert your hosts list in a lookup (e.g. perimeter.csv) and then use this lookup for serching the missed ones, note that for only 4 host isn't mandatory to use a lookup, you could insert them in your search (host=host1 OR host=host2 OR host=host3 OR host=host4), but it's a best practice!
Every way try something like this:
index=myindex sourcetype=mysourcetype earliest=-5m
| eval count=0, host=upper(host)
| append [ | inputlookup perimeter.csv | eval host=upper(host) | stats count by host ]
| stats sum(count) AS Total by host
| where Total=0
In this way you have all hosts that didn't send logs in the last 5 minutes.
You can use it in an alert scheduled every 5 minutes or, without the last row, in a dashboard panel to show all monitored hosts, highlighting the missed ones.
You can also show this panel in a table with coloured icons.
Bye.
Giuseppe
Can be done in multiple ways.
Host lookup (have a lookup uploaded with list of hosts)
|inputlookup lookup_hosts.csv|table host | join type=outer host [search index=myindex sourcetype=mysourcetype earliest=-5m | stats count by host ] | search NOT count =*
Increase the time frame.
index=myindex sourcetype=mysourcetype earliest=-15m|dedup host |table _time host |where _time < now() - 300
Hi stephenmoorhouse,
I suggest to you to insert your hosts list in a lookup (e.g. perimeter.csv) and then use this lookup for serching the missed ones, note that for only 4 host isn't mandatory to use a lookup, you could insert them in your search (host=host1 OR host=host2 OR host=host3 OR host=host4), but it's a best practice!
Every way try something like this:
index=myindex sourcetype=mysourcetype earliest=-5m
| eval count=0, host=upper(host)
| append [ | inputlookup perimeter.csv | eval host=upper(host) | stats count by host ]
| stats sum(count) AS Total by host
| where Total=0
In this way you have all hosts that didn't send logs in the last 5 minutes.
You can use it in an alert scheduled every 5 minutes or, without the last row, in a dashboard panel to show all monitored hosts, highlighting the missed ones.
You can also show this panel in a table with coloured icons.
Bye.
Giuseppe
Hi @gcusello , im currently stuck with the same issue as this but the difference is I don't use input lookup for my query. I have seen this comment of yours and it is possible to add the host in the search as you say but im not sure how can i add it to this query because im also using the same but im not using host using other field from the event logs Thanks in advance
Hi @neilfajardo15,
if you have few hosts is possible to insert them in the search, even if I always hint to use the lookup to have a more maintenable object.
so, if you have three hosts, you could run something like this:
| metasearch index=_internal
| eval host=lower(host)
| stats count BY host
| append [ | makeresults | eval host="host1", count=0 | fields host count ]
| append [ | makeresults | eval host="host2", count=0 | fields host count ]
| append [ | makeresults | eval host="host3", count=0 | fields host count ]
| stats sum(count) AS total By host
| where total=0
Ciao.
Giuseppe
P.S.: next time, please, open a new question.