Getting Data In

Props transforms.conf for source thats not playing nice

domenico_perre
Path Finder

Hi All,

I have been having significant trouble with one set of props/transforms for our environment. I have tried numerous things that I will detail below to no avail. The end result is that I want to move a specific type of event to an index. Sounds simple and I have done it for many others but this one is just plain out simple not working.

So here are the config files.
props.conf
[source::syslogind]
TRANSFORMS-SetIndexSourcetype = set_index_random

transforms.conf
[set_index_random]
REGEX = [A-Z]{1}[a-z]{2}\s+\d+\s+\d{2}:\d{2}:\d{2}.*SpecificText
DEST_KEY = _MetaData:Index
FORMAT = anotherindex

In my inputs.conf I have the following

[udp://514]
connection_host = dns
compressed = true
source = syslogind

Now this is what I have tried.

In props.conf I have had the following settings
[udp:514]
[source::udp:514]
[source::(udp:514)]

But still my data is being pushed into main bucket with the source and sourcetype of udp:514.

Splunk Output source = udp:514 sourcetype = udp:514

I have confirmed the regex is working within splunk using extract fields then seeing if all are ticked.

I am a little stumped for ideas. My last effort was to change the source on the inputs.conf to something random then apply the transforms on that. But that didn't work. Thanks in advance for your help.

0 Karma
1 Solution

woodcock
Esteemed Legend

Why are you not explicitly setting your sourcetype in inputs.conf (I highly recommend adding one)? Once you set a sourcetype, you can use that sourcetype for your stanza header inside props.conf. What kind of forwarder are you using (is it a Heavy Forwarder)? If not using a Heavy Forwarder, you need to deploy these changes to ALL of your indexers and the restart all Splunk instances before the changes will take effect. If using Heavy Forwarder, check out this link:

http://answers.splunk.com/answers/8531/routing-to-index-based-on-host-etc.html

View solution in original post

0 Karma

woodcock
Esteemed Legend

Why are you not explicitly setting your sourcetype in inputs.conf (I highly recommend adding one)? Once you set a sourcetype, you can use that sourcetype for your stanza header inside props.conf. What kind of forwarder are you using (is it a Heavy Forwarder)? If not using a Heavy Forwarder, you need to deploy these changes to ALL of your indexers and the restart all Splunk instances before the changes will take effect. If using Heavy Forwarder, check out this link:

http://answers.splunk.com/answers/8531/routing-to-index-based-on-host-etc.html

0 Karma

domenico_perre
Path Finder

I am marking your answer as correct woodcock as I checked this morning when I arrived as I was sure that the logs were going direct to the indexer. But they are in fact going to a heavy forwarder first. I updated the heavy forwarder props.conf and transforms.conf and it is now working. Thankyou so much :). I thought I was going crazy!!

0 Karma

domenico_perre
Path Finder

I should also add that the logs are being sent from a client that is not a universal forwarder. Otherwise I could apply the source at the outputs.conf.

0 Karma

domenico_perre
Path Finder

Hey woodcock I am doing an inputs.conf to a source because I don't want to specify a specific sourcetype as I have a range of different inputs coming in on this port. This is coming direct to the indexers. I do have heavy forwarders though.

0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...