I'm missing something, not sure what...I've got some GMT timestamped logs that Splunk didn't magically guess correctly for timezone/date format, so I figured I'd give it a helping hand in props.conf.
My events look like this:
20110825 000702531+0000 hostname01 daemon 13020 1594146607 1594146607 Note;FoobleWibble(50/6) user=name@domain.com:cmd=run
And I've added the following into props.conf for the directory where the files are located:
[source::/import/logs/files] MAX_TIMESTAMP_LOOKAHEAD = 25 TIME_FORMAT = %Y%m%d %H%M%S%3N%z
GNU date seems to interpret that correctly, and from the reading I've seen on strptime on linux, this should be right, but it just doesn't seem to get it.
Do I really need a transforms.conf entry for this? Or time_prefix? Since the timestamp is at the start of the line, I was thinking the time_format would be enough.
Am I missing something obvious?
First of all, I would not use source
if at all possible; I have always used sourcetype
for this. Secondly, your timestamps are 22 characters long, not 25. Try this:
[MySourceType]
MAX_TIMESTAMP_LOOKAHEAD = 22
TIME_FORMAT = %Y%m%d %H%M%S%3N%z
Also, you need to make sure that this is deployed to your indexers
, which is where timestamping occurs, not to your forwarder
.
Yeah:
1 9/8/11 1:35:33.000 PM 20110907 185959292+0000 blah blah 22415 691265462 691265462 Note;blah2 2 9/8/11 1:35:33.000 PM 20110907 185957218+0000 blah blah 22415 691265200 691265200 Note;blah123 3 9/8/11 1:35:33.000 PM 20110907 185956595+0000 blah blah 22415 691265275 691265275 Note;blah123123
From a timechart point of view, everything is timestamped at the moment of reading the file.
Log errors:
AggregatorMiningProcessor - Too many events (400K) with the same timestamp: incrementing timestamps 4 seconds into the future to insure retrievability
I would assume that MAX_TIMESTAMP_LOOKAHEAD = 22
should be enough. How is Splunk getting the date wrong, exactly? Could you show us an example?