We have thousands of forwarders installed. Our requirement is to check status of only 50-100 forwarders from thousands of forwarders on daily basis.
These selective 50-100 forwarders will be new every time. We will use CSV file to give forwarders input to search query.
Below search query we are using,
index=_internal sourcetype=splunkd group=tcpin_connections (connectionType=cooked OR connectionType=cookedSSL) fwdType=* guid=* [| inputlookup forwarders.csv | fields hostname]
| stats values(fwdType) as forwarder_type, latest(version) as version, values(arch) as arch, values(os) as os, max(_time) as last_connected by guid, hostname
| eval status = if(last_connected < (now() - 900), "missing", "active")
| eval last_connected=strftime(last_connected,"%m/%d/%y %H:%M:%S")
| fields guid, hostname, forwarder_type, version, arch, os, status, last_connected
This query is working as per the expectations. However I have one more concern i.e.
We want to see the status of forwarders as "Not Found", For those forwarders which does not exist in enviornment or For those forwarders which log entries are not present in internal index.
This query is not returning status for those forwarders which does not exist or whose log entries are not available.
I will be thankful If any of you can guide or provide search query.
Thanks
MetaWoot combined with a custom query to determine when the forwarders last sent data in would probably solve your issue...