In an inputs.conf I can define a forwarder's host field which I can use in searches. Identifying a single host is not always enough, e.g., we have build servers, source code management servers, filers etc. to monitor - a kind of grouping would come in handy in order to limit searches to a certain group of hosts. We could do that by maintaining lists and using them as look-ups, but I was wondering whether there's the possibility to achieve that on the forwarder with just configuration. Example:
inputs.conf on host A:
host=hosta
group=build_servers
host=hostb
group=git_servers
I want to be able to search for something like
search host=* group="build_servers" sourcetype="df" ... | ...
Is there a way to do this?
Splunk records pretty limited information per-event; your best bet would be to either have a lookup field (which you mentioned may not work), filters as a set of macros or eventtypes (again, based on static info), or to have it built-in to one of the default metadata-scraped fields:
I almost suspected that. Thanks for summarizing my options ... "host" and "source" won't work because those are outside my jurisdiction (I don't own the monitorees) and I agree, abusing "sourcetype" for that purpose would harm "sourcetype" as a more or less well-known concept in my Splunk deployment.