let's say the orignal filename is
/var/log/mylog.log
and the rotated files
/var/log/mylog.log.1
/var/log/mylog.log.2
/var/log/mylog.log.3.gz
/var/log/mylog.log.4.gz
...
And let's imagine that Splunk reached the line 99 of the original file, then a new line is added, and the file rotate to mylog.log.1
the solution is to monitor all the files.
[monitor:///var/log/mylog*]
Splunk will check the crc of the first 256 chars and detects that a file is a rotated version of a file that was indexed.
Then will continue where it was (on line 99), while indexing the new file. Also Splunk can read compressed files.
In your case, if the files are backed up out of the server, and rotate very often, you may want to keep some first rotated versions on disk.
see details http://docs.splunk.com/Documentation/Splunk/5.0.1/Data/HowLogFileRotationIsHandled
