Getting Data In

props.conf entries

DTERM
Contributor

I need to perform some date calculations in my app. Every entry in my logs will have multiple custom dates that I need to query. The time formats in the logs look like:

firsttime=2012/08/10 22:14:13
actiontime=2012/08/11 10:25:03
lasttime=2012/08/12 12:23:26

I've entered the following in props.conf

[lasttime]
TIME_PREFIX = lasttime
FORMAT = %Y/%m/%/d %H:%M:%S

[firsttime]
TIME_PREFIX = firsttime
FORMAT = %Y/%m/%/d %H:%M:%S

[actiontime]
TIME_PREFIX = actionTime
FORMAT = %Y/%m/%/d %H:%M:%S

Is this format accurate? If so, how to run a query for all lines where (for example) firsttime is older then 5 days?

Thanks

Tags (1)
1 Solution

dwaddle
SplunkTrust
SplunkTrust

What you have configured so far is probably not what you want. You have told Splunk how to do timestamp recognition for three different sourcetypes, lasttime, firsttime, and actiontime. I doubt that your sample event is any of these three sourcetypes, and am 100% confident that it is not all three at once.

I am going to make an assumption that your firsttime, lasttime, and actiontime are all within a single event and that, thanks to auto-kv extraction, they are already pulled out as fields. To do your comparison, the easiest thing to do is yank them into time_t values.

sourcetype=mysourcetype 
| eval firsttime_t=strptime(firsttime,"%Y/%m/%/d %H:%M:%S")
| eval lasttime_t=strptime(lasttime,"%Y/%m/%/d %H:%M:%S")
| eval actiontime_t=strptime(actiontime,"%Y/%m/%/d %H:%M:%S")
| where firsttime_t <= (now() - (86400 * 5))

View solution in original post

dwaddle
SplunkTrust
SplunkTrust

What you have configured so far is probably not what you want. You have told Splunk how to do timestamp recognition for three different sourcetypes, lasttime, firsttime, and actiontime. I doubt that your sample event is any of these three sourcetypes, and am 100% confident that it is not all three at once.

I am going to make an assumption that your firsttime, lasttime, and actiontime are all within a single event and that, thanks to auto-kv extraction, they are already pulled out as fields. To do your comparison, the easiest thing to do is yank them into time_t values.

sourcetype=mysourcetype 
| eval firsttime_t=strptime(firsttime,"%Y/%m/%/d %H:%M:%S")
| eval lasttime_t=strptime(lasttime,"%Y/%m/%/d %H:%M:%S")
| eval actiontime_t=strptime(actiontime,"%Y/%m/%/d %H:%M:%S")
| where firsttime_t <= (now() - (86400 * 5))

DTERM
Contributor

Thanks for the reply.

0 Karma

Ayn
Legend

I think you're confusing some concepts here.

First of all, stanzas in props.conf refer to what type of events something should apply to - most commonly this is a sourcetype, like in your props.conf where you have for instance a stanza [actiontime]. That's telling Splunk that the settings below the stanza should apply to the sourcetype actiontime.

Second, there is no FORMAT configuration directive in props.conf. There IS a directive called TIME_FORMAT, but that only applies to how Splunk should parse a timestamp in an event for creating its own event timestamp (_time).

If you want to make use of strftime/strptime format strings for extracting time values from timestamp strings, you should use eval's strftime and strptime functions, or possibly the convert command. More information on eval, its functions and finally convert, respectively, can be found at the following sections in the docs:

http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Eval
http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/CommonEvalFunctions
http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Convert

DTERM
Contributor

ticketNumber=1|firstTime=2012/07/25 10:20:18|lastTime=2012/07/25 09:36:17|writeTime=2012/07/25 18:23:58|actionCode=U|actionTime=2012/07/25

A single line, the rest would be the same, with different times. I'd like to be able to run a query to find all lines in a log where firstTime is older then 5 days. I know this can be accomplished using regular expressions, but I want to get away from that. Thanks.

0 Karma

lguinn2
Legend

Can you provide a few lines of one of the logs?

0 Karma

Ayn
Legend

Wait, are you saying that these events all have their own sourcetype, so the one with "firsttime" actually has sourcetype "firsttime" and so on? Because the stanzas you've defined in props.conf seem to suggest that.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...