Hello Splunkers -
I'm having trouble figuring out how to make the following work.
I get usage files from a popular CDN delivered to me via FTP. These files come in gzipped...but, Splunk is nice and handles all of that wonderfully.
However, on rare occasions, we may have the usage files redelivered from said CDN. When they are redelivered, the contents of the gzip are identical, but the modified time of the gzip is different (bytes 9-12)...causing Splunk to re-index (which doubles my counts for that period)...which is bad.
So, I'm trying to get around splunk using the first/last 256 bytes to determine uniqueness...I'd like to use something like:
CHECK_METHOD=none
crcSalt =
...which would use filename as the ONLY factor when determining uniqueness, but "CHECK_METHOD=none" isn't an options.
Can anybody suggest an alternative approach?
... View more