Getting Data In

How to configure a heavy forwarder to reroute data from a universal forwarder into a special index different from the one specified in the UF's inputs.conf?

tpaulsen
Contributor

Hi,

We need some help with rerouting data from a universal forwarder via a heavy forwarder into a different index than given in the UF inputs.conf.

Universal Forwarder inputs.conf:

[monitor:///var/opt/log/tomcat/test.log]
disabled = false
sourcetype = testdatenb2c

Heavy Forwarder (as man in the middle) configuration:
inputs.conf:

[default]
host = splunk-heavy-dev-369368
[splunktcp://9697]
connection_host = ip

props.conf:

[testdatenb2c]
TRANSFORMS-routing = transforms_TEST

transforms.conf

[transforms_TEST]
DEST_KEY=_TCP_ROUTING
FORMAT=B2CDATEN2EC
REGEX=.

outputs.conf

[tcpout:B2CDATEN2EC]
#Disable the current filters from the defaults outputs.conf
forwardedindex.0.whitelist =
forwardedindex.1.blacklist =
forwardedindex.2.whitelist =

forwardedindex.0.whitelist = test
disabled = false
server = my-splunk-idx-dev:9997

Now, the data gets indexed by the my-splunk-idx-dev server, but the data doesn't land in the index "test". Instead, it is being indexed in "main" as the default configured on the UF.
What are we doing wrong here? How can we reroute the data to our "test" index on the my-splunk-idx-dev server?

Thank you in advance for any help,

Best regards, Tom

1 Solution

somesoni2
SplunkTrust
SplunkTrust

The outputs.conf forwardedindex whitelist setting only does the filter of events based on index name but doesn't update the index name in the data. That has to be done in the props-transforms conf file.

Try this for your props.conf and transforms.conf in your Heavy forwarder
Updated fixed typo, added missing underscore

props.conf
[testdatenb2c]
TRANSFORMS-routing = transforms_routing,transforms_index

transforms.conf

[transforms_routing]
DEST_KEY=_TCP_ROUTING
FORMAT=B2CDATEN2EC
REGEX=.

[transforms_index]
DEST_KEY = _MetaData:Index
FORMAT = test
REGEX=.

View solution in original post

somesoni2
SplunkTrust
SplunkTrust

The outputs.conf forwardedindex whitelist setting only does the filter of events based on index name but doesn't update the index name in the data. That has to be done in the props-transforms conf file.

Try this for your props.conf and transforms.conf in your Heavy forwarder
Updated fixed typo, added missing underscore

props.conf
[testdatenb2c]
TRANSFORMS-routing = transforms_routing,transforms_index

transforms.conf

[transforms_routing]
DEST_KEY=_TCP_ROUTING
FORMAT=B2CDATEN2EC
REGEX=.

[transforms_index]
DEST_KEY = _MetaData:Index
FORMAT = test
REGEX=.

tpaulsen
Contributor

Thank you very much! It worked! For others...note that it´s _MetaData (with leading underscore).

0 Karma

wkupersa
Path Finder

Thanks to both of you. Very helpful!

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...