Hello!
I'm using splunk to monitor kubernetes pod log files. Which sit on the nodes, the file name is as follows:
podname_namespace_dockername.log
Within the input.conf file, I would like to dynamically label the log file before sending it up to Splunk cloud. I'd like to use the podname field as the hostname, then the namespace as the second field.
I can easily get the host section, but I'm having difficulty getting the regex match on the second part of the file name. I went through the doco and used some regex (PCRE) validators and they advised I'm doing it right.
Any help would be greatly appreciated.
My attempt:
[monitor:///var/log/containers/*.log]
host
from the first group in the filename host_regex = /var/log/containers/(.*)_.*_.*\.log
namespace
from the first group in the filename namespace = /var/log/containers/.*_(.*)_.*\.log
hey try something like this!
host_regex = /var/log/containers/([^_]+)
namespace = /var/log/containers/[^_]+_([^_]+)
I do not know what is namespace as I have not found in the docs of inputs.conf but I have written the regex for you.
Also if you want to extract host and namespace at search time then you can try this run anywhere search
| makeresults | eval source="/var/log/containers/podname_namespace_dockername.log" | rex field=source "\/var\/log\/containers\/(?<host>[^_]+)_(?<namespace>[^_]+)_.*\.log"
Let me know if this helps you!
Are you having trouble getting a regex to match, or are you having issues creating a new indexed field (like host, source, etc)?
I assumed just by specifying the regex for a new value it would create a new indexed field. But that's probably what I'm doing wrong.
Please try -
index=<any_index>
| eval _raw="/var/log/containers/podname_xxxx_dockername.log"
| rex field=_raw "/var/log/containers/(?<host>.*)_(?<namespace>.*)_.*\.log"
It extracts host
and namespace
.
Therefore, for host
your solution should work -
host_regex = /var/log/containers/(.*)_.*_.*\.log
It doesn't seem that you can do the same for namespace
.