Getting Data In

Universal Forwarder and props.conf and transforms.conf

phoenixdigital
Builder

Just a quick question regarding the "Universal Forwarder"

I have setup my inputs.conf and outputs.conf in
/opt/splunkforwarder/etc/apps/SplunkUniversalForwarder/local/
this works perfectly

However I also wanted to perform some processing on these inputs prior to sending to the indexer.

It made sense that I would need to add props.conf and transforms.conf to this directory.

This however did not appear to work. Adding the props.conf and transforms.conf files to the indexer worked however.

Is there a way to do this on the universal forwarder or does it need to be done on the indexer?

Tags (1)
1 Solution

dwaddle
SplunkTrust
SplunkTrust

Universal Forwarder and Light Forwarder do not parse events before passing them on to the indexer. Because they do not, most props.conf and transforms.conf settings need to be done at the indexer. This is what makes these two Forwarders "lighter" than the standard "Heavy" forwarder and a Splunk indexer.

View solution in original post

irwinj_125
Explorer

Apologies for my ignorance.  I've having a similar issue.

Regarding this comment:

"This however did not appear to work. Adding the props.conf and transforms.conf files to the indexer worked however."

Exactly how do you add these files to the indexer?

0 Karma

DUThibault
Contributor

You can have your Universal Forwarder do the index-time work, meaning SEDCMD and TRANSFORMS, as well as sourcetyping. The trick is that the Universal Forwarder's props.conf and transforms.conf must be on the forwarder (if you edit them in /opt/splunk/etc/deployment-apps/_server_app_<forwarder_class>/local/, Splunk will send the files to the forwarders for you) and the props.conf [<sourcetype>] and [source::<source>] stanzas must have a force_local_processing = true clause. Note that if the Universal Forwarder does the indexing, the Splunk instances won't: all of the index-time work must be done on the Universal Forwarder.

dwaddle
SplunkTrust
SplunkTrust

Universal Forwarder and Light Forwarder do not parse events before passing them on to the indexer. Because they do not, most props.conf and transforms.conf settings need to be done at the indexer. This is what makes these two Forwarders "lighter" than the standard "Heavy" forwarder and a Splunk indexer.

walterk82
Path Finder

most props.conf and transforms.conf settings need to be done at the indexer

Is there a more comprehensive definition of "most" and "works"?

0 Karma

walterk82
Path Finder
0 Karma

phoenixdigital
Builder

Thanks for the information. Makes sense from the perspective of 'light' and 'heavy' system usage.

0 Karma
Get Updates on the Splunk Community!

Join Us for Splunk University and Get Your Bootcamp Game On!

If you know, you know! Splunk University is the vibe this summer so register today for bootcamps galore ...

.conf24 | Learning Tracks for Security, Observability, Platform, and Developers!

.conf24 is taking place at The Venetian in Las Vegas from June 11 - 14. Continue reading to learn about the ...

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...