Getting Data In

Why is my props.conf configuration no longer working on my French timestamp and FileZilla server logs?

splunkreal
Motivator

Hello guys,

I have a problem with French logs so I tried to create props.conf and deploy it :

[fzs]
TIME_PREFIX = ^\([0-9]*\)\s
TIME_FORMAT = %d/%m/%Y %H:%M:%S

Log example :

(1002561) 01/04/2017 23:59:01 - blablabla

I've understood that the time_prefix will ignore the (number) and space before the french date.

Should it work? My logs from April are not coming however it worked from January to March 2017.

Thanks a lot!

* If this helps, please upvote or accept solution 🙂 *
0 Karma
1 Solution

s2_splunk
Splunk Employee
Splunk Employee

You need to deploy your props.conf settings where the event parsing happens. There are a few that take effect on the Universal Forwarder, but most of them need to be on your indexing tier (or any intermediary heavy forwarder, if present).
This Wiki page provides a bit more detail around the topic.

View solution in original post

s2_splunk
Splunk Employee
Splunk Employee

You need to deploy your props.conf settings where the event parsing happens. There are a few that take effect on the Universal Forwarder, but most of them need to be on your indexing tier (or any intermediary heavy forwarder, if present).
This Wiki page provides a bit more detail around the topic.

splunkreal
Motivator

Ok, I've tried on a test machine and it works finally (I used the add data/upload web interface with the advanced sourcetype settings).

So I've to put my props.conf on both /etc/deployment-apps/_server... and therefore etc/master-apps/_cluster/local?

Thanks a lot!

* If this helps, please upvote or accept solution 🙂 *
0 Karma

s2_splunk
Splunk Employee
Splunk Employee

Great to hear, you're welcome. Please accept answer when you get a chance. Thanks!

s2_splunk
Splunk Employee
Splunk Employee

Where did you deploy your props.conf file?
If your Filezilla logs are being collected with a Universal Forwarder, props.conf needs to be on all the indexers, if you are using a Heavy forwarder somewhere between your Filezilla server and the indexers, it needs to go on the Heavy Forwarder.

In other words: props.conf needs to be on the Splunk role that does the event parsing.

Maybe it helps if you describe your data ingest path a bit more.

0 Karma

splunkreal
Motivator

You are right, props.conf is only forwarder side. Should it be deployed on the indexer cluster?

* If this helps, please upvote or accept solution 🙂 *
0 Karma

cpetterborg
SplunkTrust
SplunkTrust

Props.conf files usually deployed on the indexers, and for the functionality that you want, that is where the props.conf should be, because it is at index time, not at forwarding time that those configs are needed.

cpetterborg
SplunkTrust
SplunkTrust

That should be working. Have you checked to make sure that there aren't extra spaces or something else that might have changed slightly in the log since April 1?

0 Karma

splunkreal
Motivator

I never used TIME_PREFIX and TIME_FORMAT before, in fact april logs are now indexed as march which is the problem :

03/04/2017 (3rd april 2017 french format) => 4 march 2017

Thanks.

* If this helps, please upvote or accept solution 🙂 *
0 Karma

cpetterborg
SplunkTrust
SplunkTrust

It looks like it is ignoring your TIME_FORMAT. Are you sure that the stanza is being used for your data? If not, then it would try to do formatting on its own, and that might make it use an American mon/day/year format, which looks like what you are seeing.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...