Do SplunkForwarder forward the metrics.log to the Splunk indexer automatically? I can see the splunkd.log files but not metrics.log file
This must have been updated with 6.2.1/6.2.2, I now see the following entry by default in "etc\apps\SplunkUniversalForwarder\default"
[monitor://$SPLUNK_HOME\var\log\splunk\metrics.log]
_TCP_ROUTING = *
index = _internal
So both splunkd.log and metrics.log are now being forwarded to _internal
I see that in the forwarder app but I also see this in etc/system/default/input.conf which appears to be sending not only the .log files but also the rolled over log files such as .log.1, .log.2, etc.
[monitor://$SPLUNK_HOME\var\log\splunk]
index = _internal
By default, universal and lightweight forwarders are not forwarding the metrics.log, only splunkd.log.
You can bypass this and force the metrics.log to be forwarded with an inputs.conf like
[monitor://$SPLUNK_HOME/var/log/splunk/metrics.log]
index=_internal
_TCP_ROUTING = *
No, the metrics.log isn't forwarded automatically. Only the splunkd.log receives a special exception. If you look at the documentation for inputs.conf here, it says explicitly:
* To forward data from the "_internal" index, _TCP_ROUTING must explicitly be set to either "*"
or a specific splunktcp target group.
The splunkd.log has this setting, but the general directory $SPLUNK_HOME/var/log/splunk does not. You'll have to create a local inputs.conf (in a small config app, or in system/local) containing:
[monitor://$SPLUNK_HOME/var/log/splunk]
_TCP_ROUTING = *
Once this is in place, restart your forwarder.