Getting Data In

Can the formatting of the timestamp added by the collect command be modified?

laserval
Communicator

I'm using collect to create a summary index of a subset of events that all have a specific field, with all field extractions done.
It basically looks like this:

index=coolstuffs where NOT isnull(piece_id) | eval rawevent=_raw | fields - _raw | collect addtime=true index=coolstuffs_piece_events`

This gives me events of the format:

11/14/2013 13:08:54 +0100, info_min_time=12324545.000, ..., some_extracted_field="value", rawevent="2013-11-14 14:08:54.067 INFO here is the raw event", piece_id="ASde123"

Which means the summary index has dropped sub-second accuracy, which breaks the ordering of the events when searching on the index. If I set addtime=false, no timestamp is added to the event (of course), which breaks the index even more as splunk can't find the timestamp because I removed _raw (necessary to get the field extractions).

Is there any way of configuring the format for the timestamp collect uses?

I'm using Splunk 6 - on Windows.

0 Karma
1 Solution

laserval
Communicator

And now I think I solved this. Apparently collect takes the timeformat option that search also has - which can change the format of the output timestamp.

I'm now doing this:

| collect timeformat="%F %T.%3Q %z" addtime=true index=...

This gives me events that look like this:

2013-11-14 13:54:55.992 +0100, search_name=...

Much more usable. I don't really see the reason for dropping sub-seconds in the default timeformat. It also seems like an undocumented behaviour.

View solution in original post

laserval
Communicator

And now I think I solved this. Apparently collect takes the timeformat option that search also has - which can change the format of the output timestamp.

I'm now doing this:

| collect timeformat="%F %T.%3Q %z" addtime=true index=...

This gives me events that look like this:

2013-11-14 13:54:55.992 +0100, search_name=...

Much more usable. I don't really see the reason for dropping sub-seconds in the default timeformat. It also seems like an undocumented behaviour.

jtrucks
Splunk Employee
Splunk Employee

mark your own answer as accepted!

--
Jesse Trucks
Minister of Magic
0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...