Knowledge Management

How to set the timestamp when using the collect command?

regriffith
Path Finder

I am searching yesterday's data and trying to insert it into an index for reporting purposes. I need to take multiple indexed events with various date/time fields and override them with the current date/time for the summary index table. The following search is a very simplified version that illustrates the issue.

index=blah

| eval _time=now()
| collect index=test

When I do the search, it inserts yesterday's date/time into the summary index _time field. Is there any way to reassign this?

Splunk 6.6.3.

Labels (1)

adobrzeniecki
Path Finder

So i figured out a way to retain _time. Whatever you are bringing over into your summary index; source, sourcetype, fields of your choice....Create your own _raw field. In my instance I created _raw as below:
| eval _raw= _time. ":" .source
| table _raw ALL OTHER FIELDS YOU WANT
| collect index=SI

This will retain the _time value in your summary index. If this works for you please upvote this response!

0 Karma

mayurr98
Super Champion

hey
you can set the value of the _time field before the collect.

Example:

... | addinfo | eval _time=info_max_time | collect index=test

you can use any eval function to calculate the _time value.

ddrillic
Ultra Champion

Maybe | collect index=test addtime=true

0 Karma

aliuwkina
Engager

It's working! But you need to use addtime=false and Splunk taking first timestamp in your event.

0 Karma

micahkemp
Champion

The docs suggest this would use info_min_time first, if present (which it sounds like it would be), and is true by default anyway:

addtime
Syntax: addtime=<bool>
Description: Use this option to specify whether to prefix a time field on to each event. Some commands return results that do not have a _raw field, such as the stats, chart, timechart commands. If you specify addtime=false, the Splunk software uses its generic date detection against fields in whatever order they happen to be in the summary rows. If you specify addtime=true, the Splunk software uses the search time range info_min_time. This time range is added by the sistats) command or _time. Splunk software adds the time field based on the first field that it finds: info_min_time, _time, or now().
Default: true
0 Karma

micahkemp
Champion

When you pipe events to collect with the _raw field present, that field will be used as the only value that gets indexed, and _time will be parsed from that text.

If you want to set your own _time, I suggest including only the fields you want and need in your summary index, like this:

index=blah | table user http_uri | eval _time=now() | collect index=test

Edit:

you could remove the old timestamp from the lines and insert a new one, but I would consider that a poor choice, as you're altering what was previously evidentiary quality data

Or perhaps you could collect the event with _raw in a new field. Try this:

index=blah | eval orig_raw=_raw | eval _time=now() | table _time orig_raw | collect index=test
Get Updates on the Splunk Community!

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...