Hi Everyone,
Our setup is a universal forwarder --> heavy forwarder --> indexer. I am looking to modify a universal forwarder config so I can search on static metadata in Splunk Web. For example, I'd like to be able to search for an app_name
, build_version
, or environment_name
that would be set when the instance comes up.
I have seen various posts on this site about accomplishing that and most of them come back to the link below. This seems like the correct path, but many of the keys are out of date. I have finally settled on the structure below for my files, but I am not seeing anything in Splunk Web. Is this outcome just not possible with Splunk, or am I missing something?
props.conf:
[host::i-e420f63c]
TRANSFORMS-test = MYTRANSFORM
transforms.conf:
[MYTRANSFORM]
REGEX = .*?
SOURCE_KEY = _raw
FORMAT = instance::app_name
I can suggest one workaround using which you can send/set multiple values that can be later used for categorization of the data coming from the forwarders. What you can do is that you can set value of host (this can be set in inputs.conf
on UF) to a combination of whatever values you want to assign, say concatenated by colon or something. They will be set on Client .On indexer side, you extract those categorization fields and put into separate fields which you can use during searching/reporting.
So, on inputs.conf (on UF)
[monitor://foo/bar/something.log]
index=blah
sourcetype=blahblah
host=customPrefix:field1:field2:field3
For index time field extraction, put on HeavyForwarder/indexer, else on Search Head
props.conf
[host:customPrefix*]
REPORT-extractcustom = customfields ***User this for search time field extractions
TRANSFORMS-extractcustom = customfields ***Use this for index time field extraction
transforms.conf
[customfields]
SOURCE_KEY = MetaData:Host
REGEX = customPrefix:(?<Field1>\w+):(?<Field2>\w+):(?<Field3>\w+)
Update
Well here, you are passing a pre-defined list of fields (just values but in specific order) with a custom prefix so that all hosts following that pattern can be identified during extraction even though they send data for different sourcetypes) from the forwarder using inputs.conf entry. On Search Head (since you just want search time extraction, I would use that term only), the transform is applied for all host with custom prefix and extract the field, again in specific order in which they were sent. E.g. if you want to send app_name, build_number and environment_name, you'll pass host as trythis:myapp:12345:production
. On your transforms, you'll extract them using REGEX = trythis:(?.*):(?.*):(?.*)
.
Again, the REGEX offers so many things that you can pass dynamic key value pairs as well. Like this
inputs.conf (on UF) host=trythis:#app_name=myapp#:#build_number=12345#:#environment_name=production#
transforms.conf (on SH) REGEX = #(?<_KEY_1>[^=]+)=(?<_VAL_1>[^#]+)
So I don't want to do anything that will negatively impact index performance. Search-time seems like the best option but how is this different than just searching for those values in the host name? What exactly is the transforms.conf doing in your example? Ideally, I am passing name/value pairs (e.g. app_name=myapp). Does your structure offer anything more?
Well here, you are passing a pre-defined list of fields (just values but in specific order) with a custom prefix so that all hosts following that pattern can be identified during extraction even though they send data for different sourcetypes) from the forwarder using inputs.conf entry. On Search Head (since you just want search time extraction, I would use that term only), the transform is applied for all host with custom prefix and extract the field, again in specific order in which they were sent. E.g. if you want to send app_name, build_number and environment_name, you'll pass host as trythis:myapp:12345:production
. On your transforms, you'll extract them using REGEX = trythis:(?.*):(?.*):(?.*)
.
Again, the REGEX offers so many things that you can pass dynamic key value pairs as well. Like this
inputs.conf (on UF) host=trythis:#app_name=myapp#:#build_number=12345#:#environment_name=production#
transforms.conf (on SH) REGEX = #(?<_KEY_1>[^=]+)=(?<_VAL_1>[^#]+)
This transform should happen on Heavy forwarder and check the REGEX and FORMAT (not matching with the post https://answers.splunk.com/answers/39405/adding-static-field-value-using-props-transforms-based-on-s...)
@somesoni2 what should the REGEX and FORMAT be? I have tried the syntax from the previous post and that doesn't work either. These values would be set at instance creation time and are not extracting from anything else.
Also, why is the heavy forwarder the only option? We have hundreds of instances that come and go so adding this to a HF doesn't sound like a feasible option.