Splunk Search

Why am I getting error "Could not use regex to parse timestamp..." for timestamps before November 2010?

TangentTexan
New Member

Using Splunk 6.4.0 on Ubuntu Server

Trying to index a file that goes back in years. Working with the Timestamp to get it indexed correctly - I ran into a problem with it for Time Stamps before 11-30-2010, using this format: %Y-%m-%d %H:%M:%S %Z

Am I missing something? Thank you.

Sample Data: timedate, number


Could not use regex to parse timestamp from "2010-11-24 22:00:00 EST".
2010-11-24 22:00:00 EST,3.1
Could not use regex to parse timestamp from "2010-11-25 22:00:00 EST".
2010-11-25 22:00:00 EST,2.22
Could not use regex to parse timestamp from "2010-11-26 22:00:00 EST".
2010-11-26 22:00:00 EST,3.33
Could not use regex to parse timestamp from "2010-11-27 22:00:00 EST".
2010-11-27 22:00:00 EST,4.44
Could not use regex to parse timestamp from "2010-11-28 22:00:00 EST".
2010-11-28 22:00:00 EST,5.2
Could not use regex to parse timestamp from "2010-11-29 22:00:00 EST".
2010-11-29 22:00:00 EST,6.1
2010-11-30 22:00:00 EST,7.2
2010-12-01 22:00:00 EST,8.5
2010-12-02 22:00:00 EST,9.8
2010-12-03 22:00:00 EST,9.2
2010-12-04 22:00:00 EST,9.2
2010-12-05 22:00:00 EST,9.9

0 Karma

antlefebvre
Communicator

I know this is a dated question, but ran into the same issue. The solution for me was to increase MAX_DAYS_AGO in props.conf. Looks like based on your description of it not working for 6 year old data this was your issue as well. Since Splunk by default ignores dates over 5 1/2ish years old.

From the docs site:

Specifies the maximum number of days in the past, from the current date, that an extracted date can be valid.
For example, if MAX_DAYS_AGO = 10, Splunk software ignores dates older than 10 days from the current date and instead either uses the timestamp of the previous event, or uses the current index time of the event if it cannot determine a timestamp in the previous event.

The maximum settable number of days in the past is 10951.

Defaults to 2000 days
Note: If you have data that is more than 2000 days old, increase this setting.

ddrillic
Ultra Champion

Looks fine to me -

base search 
| eval CREATION_DATE="2010-11-24 22:00:00 EST"
| eval xxxx=strptime(CREATION_DATE,"%Y-%m-%d %H:%M:%S %Z") 

returns xxxx as 1290654000.000000 which is Thu, 25 Nov 2010 03:00:00 GMT

0 Karma

TangentTexan
New Member

I was trying to process the time at index time as I am trying to do a fresh index data. as new data gets entered this is not an issue. Just the older data.

Thanks

0 Karma

woodcock
Esteemed Legend

Many times this is because the spacing between the date elements changed at some point. This is very hard to see and also a big pain because it requires you to switch from using the "easy" configurations to using datetime.xml (so that you can specify more than one time format. Very carefully check the spacing and you will probably find variation.

0 Karma

TangentTexan
New Member

I went through the standard process of verifying the data for any special characters or spaces... it was a no go.
I simplified the CSV and found all dates before 11-30-2010 would not get assigned the time stamp.
I think this is a bug. So I ended up excluding all log data before 1-1-2011 and stopped battling the timestamp. 🙂
I appreciate your time - thanks!

timedate,num
11/24/1996 22:00:00,1
11/25/1999 22:00:00,2
11/26/2000 22:00:00,3
11/27/2010 22:00:00,4
11/28/2010 22:00:00,5
11/29/2010 22:00:00,6
11/30/2010 22:00:00,7
12/01/2010 22:00:00,8
12/02/2010 22:00:00,9
12/03/2010 22:00:00,0
12/04/2010 22:00:00,10
12/05/2010 22:00:00,11

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...