Getting Data In

Sourcetype assigning

RobertRi
Communicator

Hello

I have troubles asigning sourcetypes for multiple filetypes in one directory. I have read a few posts which talk about the same problem, but I don't get the right solution to get it work

I have a directroy with two filetypes servername_app1_timestamp.log and servername_app2_timestamp.log

Now I want to asign servername_app1_.... the sourcetype app1 ans so on.

This is my inputs.conf on the forwarder
[monitor:///applications/logs/]
disabled = false
alwaysOpenFile = 1
whitelist = servername(app1|app2)_.*.log$
index = myapps
sourcetype = default_apps

and this the props.conf from the indexer
[source:.../servername_app1_*]
sourcetype = app1

[source:.../servername_app2_*]
sourcetype = app2

We use version 4.1.4. Do you have a clue whats going wrong

Thanks Rob

Tags (2)
0 Karma
1 Solution

RobertRi
Communicator

My solution was the following

I have only configured the forwarder (not a lightweightforwarder) If I understand it right, than a normal forwarder made the parsing of the messages before he sends the data to the indexer and so I can configure the props.conf on the forwarder.

inputs.conf
[monitor:///applications/logs]
disabled = false
whitelist = servername(app1|app2)_.*.log$
alwaysOpenFile = 1
index = default_apps

props.conf
[source::.../servername_app1_....log]
sourcetype = app1

[source::.../servername_app2_....log]
sourcetype = app2

View solution in original post

0 Karma

RobertRi
Communicator

My solution was the following

I have only configured the forwarder (not a lightweightforwarder) If I understand it right, than a normal forwarder made the parsing of the messages before he sends the data to the indexer and so I can configure the props.conf on the forwarder.

inputs.conf
[monitor:///applications/logs]
disabled = false
whitelist = servername(app1|app2)_.*.log$
alwaysOpenFile = 1
index = default_apps

props.conf
[source::.../servername_app1_....log]
sourcetype = app1

[source::.../servername_app2_....log]
sourcetype = app2

0 Karma

dskillman
Splunk Employee
Splunk Employee

Since you are manually setting the sourcetype anyway, try setting it at the input rather than via props.conf. Monitor works for Files or Directories.

You should be able to have two inputs:

[monitor:///applications/logs/servername_app1*]
disabled = false
followTail = 0
index = main
sourcetype = app1

[monitor:///applications/logs/servername_app2*]
disabled = false
followTail = 0
index = main
sourcetype = app2

And no need for the props.conf entries.

0 Karma

dskillman
Splunk Employee
Splunk Employee

According to the Dev team, the settings above should work in 4.1. If they don't it's bug. I will work on getting the docs updated.

0 Karma

RobertRi
Communicator

This is the link where the info about overlapping input stanzas is available http://www.splunk.com/base/Documentation/latest/Admin/Specifyinputpathswithwildcards

0 Karma

dskillman
Splunk Employee
Splunk Employee

I believe it used to be true, but AFAIK 4.1 should have fixed this. Could you send the link to the docs you are using. May need to tweak them.

0 Karma

RobertRi
Communicator

This doesn't work. It is also described in the technical documentation and its recommended to use the props.conf to split the different logfiles from one directory.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...