Greetings everyone.
Right now I am working with a filetype which contains a compilation of events from 4 different sources. Three of those original sources have fairly precise timestamps (to milliseconds), whereas one does not - it is precise to the second level. So what you could see in the timestamp field is:
Source 1: 20110101235959999
Source 2: 20110101235959999
Source 3: 20110101235959999
Source 4: 20110101235959
In my props.conf here are the settings:
TIME_FORMAT=%Y%m%d%H%M%S%3N
TIME_PREFIX=^([^,]*,){24}
TZ=UTC
MAX_TIMESTAMP_LOOKAHEAD=20
This data will be going into a heavy forwarder and I had hoped to send everything into the same index and same sourcetype. However when I run through the timestamp preview (using 4.3) it's saying that, for source 4, strptime() is unable to obtain the timestamp. Despite this message, it looks like it's still able to ascertain the timestamp. How is splunk doing that?
Is splunk still using strptime up to the point where it hits %3N? Or is it failing back to trying to figure out the timestamp on its own, and guessing correctly?
So, from here there are a few ways I could go - I could forego the TIME_FORMAT and let splunk figure it out, since it appears to be doing a pretty fine job of that - at least with my test data. I could also set up routing on the heavy forwarder to send the less precise source to another sourcetype. And finally I could just keep everyting the way it is, which covers 3 of the sources.
For the sources where we have precision I need to keep it, since this is to help calculate latency and every ms is important.
