I have about 50 different tokens. I want data from one particular token to get some metadata added to it. Unfortunately, it doesn't appear that the _meta
directive works for http in inputs.conf. Is it possible to replicate this functionality some how?
You can use method i have implemented with DS distributed bash script automation, which does following with every single HEC input on each server in hfw pool:
First, append existing http stanzas in inputs.conf with "fake" output group, like
outputgroup = out01
Define those fake outputs in outputs conf like this:
[tcpgroup:out01]
server=127.0.0.1:9001
Now we need to set some listener on internal loop input dedicated port that "tags" the data:
[splunktcp://9001]
_meta = HecName::192.168.0.1:hec_input_1
Repeat all this for for all your hec inputs, make each of it have it's own outputgroup and tcpsplunk port listener, restart splunk and enjoy:
|tstats count where index=hec_index by HecName
The inputs name will translate into a source::http:InputNameHere
which in turn should be useable in props.conf
But I must admit, I have not yet tried it 😉
cheers, MuS
Well shoot. If the sending application sets source, that overrides the default above, which means the transform doesn't fire. So still back to the old problem: How to guarantee a transform gets applied to every single event that came through a particular token's input def?
In this case, did someone say cough cribl cough 😉
We're testing it, but not ready to roll into production. Yet. 🙂 Very promising!
Perfect! I had no idea that was a thing. I feel like I gained a new superpower.
Glad I could help - Enjoy the new superpower 🙂
cheers, MuS
Hi twinspop,
you can always use the good old props.conf / transforms.conf approach and add a meta field this way. Here is an example transforms.conf I use to add the hostname of the parsing HWF to events:
[add-relay-info-to-meta]
FORMAT = splunk_hwf::HostNameHere
REGEX = .
WRITE_META = true
Yes, it is a static value but I assume you will not change your HEC input too often 😉
Hope this helps ...
cheers, MuS
Yeah, a transform is where i was headed, but I don't see any foolproof way to identify only those those logs, and ALL those logs, that originate on 1 particular token. The token value and the input name are not things I can key off of in props as far as i know.