I am missing logs. My logs rotate faster than 5 minutes, anywhere greater than 1 min.
It seems that every 5 minutes the current log gets loaded, but when it rotates, the new file does not get loaded. If that file gets rotated before the 5 minute mark, then it gets missed.
Has anyone ran into this? is there a work around?
This is on a Universal Forward V5.0.4
Monitor both the log file and the rotated versions. Splunk can detect that they are rotated versions and finish them once rotated.
example, if your logs are :
/var/log/mylog.log
/var/log/mylog.log.1
/var/log/mylog.log.2
/var/log/mylog.log.3.gz
/var/log/mylog.log.4.gz
in inputs.conf
[monitor:///var/log/mylog*]
sourcetype=mylogsourcetype
Monitor both the log file and the rotated versions. Splunk can detect that they are rotated versions and finish them once rotated.
example, if your logs are :
/var/log/mylog.log
/var/log/mylog.log.1
/var/log/mylog.log.2
/var/log/mylog.log.3.gz
/var/log/mylog.log.4.gz
in inputs.conf
[monitor:///var/log/mylog*]
sourcetype=mylogsourcetype
Just a remark about speed, if you are using the Universal or Lighweight forwarder, they have a default limit of 256KBps, you can speed them up.
see http://answers.splunk.com/answers/78615/will-this-limit-this-forwarding-speed-to-the-indexer
yannK, thanks for your help. It seems that there is a major network issue.
Ok, so your log rotation system is doing unconventional things , and splunk considers that the files are different.
- if it's logrotate, is it using copy-truncate ?
- or is it rewriting the first lines of the log while rotating it ?
or are you using the options crcSalt or crcLength ?
Did that, and I had multipule copies of my data in Splunk.