Getting Data In

How do I edit my props.conf for proper timestamp extraction from my sample log entry?

mark19632
New Member

I'm having trouble with a log and getting Splunk to recognize the time format.

Here is an example a log entry:

010406:00:530000000000000040RD000001071215 

Now, all the entries start with 0104 followed by the time in H:M:S format.

I've added a props.conf to the indexer like this:

[sisfeedlog]
TIME_PREFIX = ^0104
TIME_FORMAT = %H:%M:%S
MAX_TIMESTAMP_LOOKAHEAD = 8

and a props.conf on the server:

[source::...\\SISFeed\\S(\d+\.LOG)]
sourcetype = sisfeedlog

It doesn't seem to be working though as the time isn't being extracted and the sourcetype is coming up as unknown.

Any advise on why it's not working?

Thanks,

Mark

0 Karma
1 Solution

MuS
SplunkTrust
SplunkTrust

Hi mark19632,

when defining a stanza in props.conf you can only use a limited set of regexes:

When setting a [<spec>] stanza, you can use the following regex-type syntax:
... recurses through directories until the match is met
    or equivalently, matches any number of characters.
*   matches anything but the path separator 0 or more times.
    The path separator is '/' on unix, or '\' on windows.
    Intended to match a partial or complete directory or filename.
|   is equivalent to 'or'
( ) are used to limit scope of |.

So, your [source::...\\SISFeed\\S(\d+\.LOG)] will not work since \d+ is not supported.
What would work, is this [source::...\\SISFeed\\S*.LOG] but I don't know if this is matching the correct directory..... BTW why don't you set the sourcetype in the servers inputs.conf?

Hope this helps ...

cheers, MuS

PS: your props.conf on the indexer looks fine.

View solution in original post

0 Karma

MuS
SplunkTrust
SplunkTrust

Hi mark19632,

when defining a stanza in props.conf you can only use a limited set of regexes:

When setting a [<spec>] stanza, you can use the following regex-type syntax:
... recurses through directories until the match is met
    or equivalently, matches any number of characters.
*   matches anything but the path separator 0 or more times.
    The path separator is '/' on unix, or '\' on windows.
    Intended to match a partial or complete directory or filename.
|   is equivalent to 'or'
( ) are used to limit scope of |.

So, your [source::...\\SISFeed\\S(\d+\.LOG)] will not work since \d+ is not supported.
What would work, is this [source::...\\SISFeed\\S*.LOG] but I don't know if this is matching the correct directory..... BTW why don't you set the sourcetype in the servers inputs.conf?

Hope this helps ...

cheers, MuS

PS: your props.conf on the indexer looks fine.

0 Karma

mark19632
New Member

Thanks MUS, I've added it to the inputs.conf and picking up the sourcetype and the time correctly 🙂

Now for the next problem!

The log files are named s*date*.log eg S15122015 .log for today.

When read in the logs are showing as the incorrect date, so 06122015 is being picked up as 12th of July rather than the 6th of December.

Can I change this?

Thanks,

Mark

0 Karma

mark19632
New Member

I've tried that and it's made no difference unfortuantly

0 Karma

mark19632
New Member

I fixed it by creating a custom datetime.xml in the end 🙂

0 Karma

MuS
SplunkTrust
SplunkTrust

use the TIME_FORMAT = option in props.conf to set the format of your time string to something like this

 TIME_FORMAT = %d%m%Y

see the docs for more details about the time strings http://docs.splunk.com/Documentation/Splunk/6.3.1/SearchReference/Commontimeformatvariables and the props.conf setting http://docs.splunk.com/Documentation/Splunk/6.3.1/Admin/Propsconf

0 Karma
Get Updates on the Splunk Community!

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...