Is there a way to check for a file from n number of hosts and alert if there is no file from each host in last 30min?

[sai_john]

index=test File="*.txt" | stats count by host | where count<1 -->with this I am getting NoResults found" but I need count 0 if there is no file from host1 or host2 or host3 or so on and need to show for which host there is no file

HostName           count
host1              0
host2              0
host3              0
:
:
hostn              0

[mpreddy · Accepted solution]

try something like this,

index=test File="*.txt" | stats count by host| search count>0 | append [|stats count | eval host="host1"] | append [|stats count | eval host="host2"] | stats max(count) as count  by host | where count=0 

This will help you for two host, in-case of multiple host you have to go with lookup to keep the list of host and change the second where condition (where n_records=2) as number of host you have.

Hope this will helps you.

[sai_john]

@mpreddy

This is working for me. Thankyou

[gcusello]

Hi sai_john,
you should create a lookup with all your server to monitor (e.g. Perimeter.csv) and run a search like this:

| inputlookup Perimeter.csv 
| eval count=0, host=upper(host)
| append [ search 
   index=_internal
   | host=upper(host)
   | stats count by host
   ]
| stats sum(count) AS Totale by host
| where Total=0

In this way all the results with Total=0 are missing and the ones with Total>0 are present.
With this search you can run an alert or to shor your infrastructure situation, maybe adding some other commands:

| inputlookup Perimeter.csv 
| eval count=0, host=upper(host)
| append [ search 
   index=_internal
   | host=upper(host)
   | stats count by host
   ]
| stats sum(count) AS Totale by host
| rangemap field=Somma severe=0-0 low=1-1000000000 default=severe 

You can graphically display your situation adding to your app two files ($SPLUNK_HOME/etc/apps/your_app/appserver/static:

• table_icons_rangemap5.js;
• table_decorations2.css

(you can take them from the Splunk Dashboard Examples App (https://splunkbase.splunk.com/app/1603/).

and adding to your dashboard the first row

<form script="table_icons_rangemap.js" stylesheet="table_decorations.css">

Bye.
Giuseppe

[DalJeanis]

The below link gives code for your answer, but here is the explanation -

When you do "| stats count by host", splunk is adding up all the events that HAVE been received for each host within your time window.

There are no records to tell splunk about any hosts that have NOT reported.

So, you either have to keep a list of which hosts you care about - the first method at the linked answer... and report the ones on the list that aren't in your summary stats...

OR you have to look at ALL the responses, and see for which hosts the latest event is earlier than your desired half hour window - the second method at the linked answer.

[jplumsdaine22]

See answer here: https://answers.splunk.com/answers/492077/stats-not-returning-zero-counts.html#answer-494838

[sai_john]

Sorry i am too early to accept this, I haven't recognized that this search which I am getting count for field "File" is not getting correct values.

Here is my search for alerting if there is no ".txt" File in last 1hr either from host1 or host2

| tstats count WHERE index=xxx earliest=-1h by _time host
| fields _time host
| join type=left host [
search index=xxx (host=host1 OR host=host2) sourcetype=abc File="*.txt"
| stats count by _time host File
]
| fillnull count value=0 | where count=0

[sai_john]

@jplumsdaine22 your tstats worked for me after little modifications.Thanks
Your Search:

| tstats earliest=-30d count WHERE index=xxx by host 
 | fields host 
 | join type=left host [ 
   search index=xxx earliest=-60m
  | bucket _time span=3m 
   | stats count by _time host IP
   ] 
 | fillnull count value=0

Modified search:

| tstats  count WHERE index=xxx earliest=-1h by _time host 
 | fields host 
 | join type=left host [ 
   search index=xxx earliest=-60m
  | bucket _time span=3m 
   | stats count by _time host IP
   ] 
 | fillnull count value=0

[jplumsdaine22]

Great! Would mind accepting the answer?

Thanks