How to parse out milliseconds from timestamp in log file?

[nitrogaute]

BREAK_ONLY_BEFORE=\d{7}
NO_BINARY_CHECK=1
SHOULD_LINEMERGE=true
TIME_FORMAT=%3N
TIME_PREFIX=\d{7}

Trying to parse out the millisecond timestamp from this log file, 9281736 :

  9281736 : COUNT IN 1003
Tx:   01 04 00 71 00 02 21 d0                              ...q..!.
Rx:   01 04 04 00 08 0a 28 7c f8    
  9282136 : COUNT IN 1003
Tx:   01 04 00 c9 00 02 a1 f5                              ........
Rx:   01 04 04 00 08 00 00 7a 46 

I suspect my TIME_FORMAT is wrong, because it breaks up events correctly with regex \d{7}.
But according to the error message it doesn't look for the timestamp in the correct spot. Or is it?

Could not use strptime to parse timestamp from ": COUNT 1003\n ......
Make sure a prefix pattern is
specified if the events don`t begin
with a timestamp.

Failed to parse timestamp. Defaulting to file modtime.

Any suggestion would be much appreciated!

[richgalloway]

Remove the TIME_PREFIX statement. That tells Splunk to start looking for a timestamp after finding 7 digits.

[nitrogaute]

That makes the timestamp appear in the in the error message. No changes appart from that in Data preview.

Could not use strptime to parse
timestamp from "9281736 : COUNT 1003n
...... Make sure a prefix pattern is
specified if the events don`t begin
with a timestamp. Failed to parse
timestamp. Defaulting to file modtime.

[richgalloway]

At least now Splunk is looking in the right place for the timestamp. Did you insert the asterisks or are they part of the data?

Tell us more about your data. What does it represent? Milliseconds since when?

[nitrogaute]

No, the asterisks is not part of the data. See the logfile entries in the above.

The logfile represent modbus communications between a master and a slave. The milliseconds is since last restart of the master node. I want to trend out timeout periods of the communications and look at what happens just before a period of timeout/no RX.

I`m also interested in other ways to do this. The logfiles are aprox 250 mb

[richgalloway]

Without a fixed reference point, I don't see how Splunk can convert your milliseconds into timestamps. Perhaps someone else will have a suggestion.

[nitrogaute]

It knows the first entry. Should`t that be OK?

[richgalloway]

Each event is processed independently and is unaware of what was processed previously.
Maybe you could write a script to process your log files and convert the milliseconds into timestamps that Splunk can process.

[nitrogaute]

I could convert milliseconds to hours, minuttes and seconds.. But would that make any difference? Or do I need the date also to give Splunk sufficient reference?

I would assume Splunk sets the reference point at 0 in a 7 digit milisecond timestamp..

[richgalloway]

Given only a time, Splunk will use the current date.

Assuming you could get Splunk to understand a 7 digit millisecond timestamp (I failed to do so), it would likely use zero as its reference point and you would end up with a timestamp in early 1970.

[nitrogaute]

Hmm, I see youre point. But I still think its strange that Splunk does not handle relative time...