Splunk Search

How can I extract a timestamp from a specific CSV field regardless of format?

j2bohan
New Member

Hi all,

I have a CSV file that could look like this:

Ticket-ID,User ID,Site ID,Site City,Site State,Create_date,Modified-date,...
TICKET,USERID,3331,CITY,STATE,3/23/2016,1459430427,...

(Note that I have two dates back to back)
or, it could look like this:

Ticket-ID,User ID,Site ID,Site City,Site State,Create_date,Modified-date,...
TICKET,USERID,3331,CITY,STATE,1459430427,1459430427,...

(Note that both Create_date and Modified-date are epoch timestamps in this version).

I unfortunately have very little control over whether (a) or (b) is handed to the forwarder.

When the file is indexed, all the records are using the create date of the CSV file rather than the Create_date field.

My props.conf looks like this:

[remedy_merch]
TIMESTAMP_FIELDS=Create_date
SHOULD_LINEMERGE=false
INDEXED_EXTRACTIONS=csv
NO_BINARY_CHECK=true
KV_MODE=none
disabled=false
pulldown_type=true
TRUNCATE=0

Through looking at other answers, I have determined that I probably need to add the following field:

TIME_PREFIX=^([^,]*,){5}

so that I always start the time "search" at the right spot in the file. However, I need to exclude Modified-date from the _time attribute, and I cannot be sure what format the Create_date will come in as. Do I have options (other than pre-processing the CSV to give it consistent timestamps)?

Thanks!

0 Karma

Richfez
SplunkTrust
SplunkTrust

For indexed_extractions, you should be able to use

TIMESTAMP_FIELDS = field1,field2,...,fieldn

Like, perhaps in your case

TIMESTAMP_FIELDS = Create_date

As per the docs on Extracting fields from files with structured data.

Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...