Getting Data In

Problems with timestamp detection with log2timeline output

mpo
New Member

Hi there,

I'm trying to import a log2timeline output (csv) into splunk, but timestamp detection fails, when I try to define a new sourcetype.

The time information is located at the line start an in the following format:
01/01/1970,00:00:00,UTC,...

I've already set the format for strptime to %m/%d/%Y,%H:%M:%S an tried automatic timestamp search as well as defining a regex.
But the only string which is highlighted (and detected as potential timestamp) ist 00:00:00.

Ipython tells that the format sting matches the timestamp:

In [11]: time.strptime("01/01/1970,00:00:00","%m/%d/%Y,%H:%M:%S")
Out[11]: time.struct_time(tm_year=1970, tm_mon=1, tm_mday=1, tm_hour=0, tm_min=0, tm_sec=0, tm_wday=3, tm_yday=1, tm_isdst=-1)

I also tried the following things:

  • Removing the "," between date and time
  • Adding a prefix to each line
  • Changing the dates to a day later than 01/01/1970
  • Swapping %m and %d just in case of any doubts

Installed Splunk version is 6.1.1

Does anybody have an idea what I can do that splunk correctly recognizes the time of the lines?
Kind regards in advance!
Markus

First lines of my log are:

01/01/1970,00:00:00,UTC,..C.,REG,NTUSER key : Typed Paths,Last Written,-,xxx,[\Software\Microsoft\Windows\CurrentVersion\Explorer\TypedPaths]...
01/01/1970,00:00:00,UTC,..C.,REG,NTUSER key : RDP Connection,Last Written,-,xxx,[\Software\Microsoft\Terminal Server Client\Default] MRU1:...
01/01/1970,00:00:00,UTC,..C.,REG,NTUSER key : Typed Paths,Last Written,-,xxx,[\Software\Microsoft\Windows\CurrentVersion\Explorer\TypedPaths] ...

0 Karma
1 Solution

jeremiahc4
Builder

If those dates are your literal dates, they are beyond the maximum number of days back Splunk will recognize (MAX_DAYS_AGO). You need to modify that max in props.conf in order to catch something that far back.

According to the documentation the maximum this number can be is 10951 though, I'm thinking that will only get you back around 1984.

http://docs.splunk.com/Documentation/Splunk/6.1.1/Admin/Propsconf

View solution in original post

0 Karma

jeremiahc4
Builder

If those dates are your literal dates, they are beyond the maximum number of days back Splunk will recognize (MAX_DAYS_AGO). You need to modify that max in props.conf in order to catch something that far back.

According to the documentation the maximum this number can be is 10951 though, I'm thinking that will only get you back around 1984.

http://docs.splunk.com/Documentation/Splunk/6.1.1/Admin/Propsconf

0 Karma

jeremiahc4
Builder

It looks like DATETIME_CONFIG could be used to mess with what gets used in the case when Splunk doesn't recognize a date. . It appears to point to a datetime.xml file which is used to recognize your timestamp. Perhaps if you're XML savvy, you could make mods there. I've not played with this before though, so can't recommend anything with it. As with any edits, make a backup copy first!

0 Karma

mpo
New Member

Weird!
Not quite a new issue - but I never thought, that the MAY_DAYS_AGO setting might be there and causes the trouble.

Thanks a lot!

Is there any additional attribute for props.conf, which sets a custom fallback time, when no timestamp is detected? I don't find any in the documentation and in answers.splunk.com.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...