We have a splunkforwarder DaemonSet in Kubernetes, which is forwarding node logs to our splunk server.
We want to take the STDOUT logs from each container, located in /var/log/containers/*.log, and index by the namespace specified in the filename. Is there a way to do this?
Filenames look as follows:
/var/log/containers/<pod-name>_<namespace>_<some-hash>.log
We'd like to set the index in inputs.conf by extracting the middle namespace from these files. I know there is a host_regex that will dynamically set the host, but I haven't found an equivalent for index.
transforms.conf:
[indexfromsource]
SOURCE_KEY = MetaData:Source
DEST_KEY = _MetaData:Index
REGEX = /var/log/containers/<pod-name>_(<namespace>)_<some-hash>\.log
FORMAT = $1
props.conf:
[<sourcetype name>]
TRANSFORMS-indexfromsource = indexfromsource
Note: the regex
is not valid, as I don't know how <pod-name>
, <namespace>
, <some-hash>
will be formatted.
transforms.conf:
[indexfromsource]
SOURCE_KEY = MetaData:Source
DEST_KEY = _MetaData:Index
REGEX = /var/log/containers/<pod-name>_(<namespace>)_<some-hash>\.log
FORMAT = $1
props.conf:
[<sourcetype name>]
TRANSFORMS-indexfromsource = indexfromsource
Note: the regex
is not valid, as I don't know how <pod-name>
, <namespace>
, <some-hash>
will be formatted.
this is good for the server when it gets the data, but is there a way to change the index on the universal forwarder side?
There is not. You would need to place this configuration on the first heavy forwarder or indexer that sees the data.
You probably want the solution found here:
https://answers.splunk.com/answers/145389/routing-data-to-specific-index-based-on-filename.html