I am indexing apache logs and have them rotating on a frequent basis. The log rotation will rename the file to error_log.1 and so forth...
I have noticed that some of my sourcetypes end up with a "-1" or "-2" at the end. For example, I have specified sourcetype=apache_error in my inputs.conf. However, I have noticed that I have some random "apache_error-2" and "apache_error-1" sourcetypes in my index. Why is this occurring?
My inputs.conf looks like this:
[source::.../var/log/httpd/error_log]
sourcetype = apache_error
In this scenario, there is the possibility that Splunk may try to index already rotated log files. This can especially occur if you have a forwarder that is turned off and the log file gets rotated multiple times. For this scenario, you can simply add a regex that recognizes the additional digit. Since Splunk performs a CRC check against the files indexed, it should not re-index old data. The proper inputs.conf stanza would look as follows:
[source::.../var/log/httpd/error_log(.\d+)?]
sourcetype = apache_error
In this scenario, there is the possibility that Splunk may try to index already rotated log files. This can especially occur if you have a forwarder that is turned off and the log file gets rotated multiple times. For this scenario, you can simply add a regex that recognizes the additional digit. Since Splunk performs a CRC check against the files indexed, it should not re-index old data. The proper inputs.conf stanza would look as follows:
[source::.../var/log/httpd/error_log(.\d+)?]
sourcetype = apache_error
you said inputs.conf in your original description - may wanna change that to props.conf 🙂