Getting Data In

How to make transforms.conf to pick from the default group of outputs.conf

premg
Engager

Hi,

We need to forward all events to indexer group_A and filtered events to indexer group_B.

We are applying some transformations on a data.
We are forwarding this data in such a way that the data with transformations are sent to server 'A' and the data without transformations are sent to server 'B'.

The props, transforms and outputs are given as follows.

props.conf

[sourcetype]
TRANSFORMS-routing=routeAllEvents,routeFilteredEvents

transforms.conf

[routeAllEvents]
REGEX=(.)
DEST_KEY=_TCP_ROUTING
FORMAT=indexers_group_A

[routeFilteredEvents]
REGEX=(some_filter)
DEST_KEY=_TCP_ROUTING
FORMAT=indexers_group_A,indexers_group_B

outputs.conf

[tcpout]
forwardedindex.filter.disable=true
defaultGroup=indexers_group_A

[tcpout:indexers_group_A]
server=server1:9997
disabled=false
autoLB=true

[tcpout:indexers_group_B]
server=server2:9997
disabled=false
autoLB=true

[tcpout:indexers_group_C]
server=server3:9997
disabled=false
autoLB=true

This works fine.

Question:

If I want to forward all events (default group) to Indexers_group_C instead of indexer_group_A, I had to change in both outputs.conf and transforms.conf.
Is there any way to make transforms.conf to pick from the default group of outputs.conf?

Tags (2)
0 Karma
1 Solution

MuS
Legend

Hi premg,

Basically you set a default destination for your data in transforms.conf and/or in outputs.conf. If you want to manage default destinations in one .conf file, do it in transforms.conf, because _TCP_ROUTING from transforms.conf will overwrite the outputs.conf defaultGroup stanza.

hope this helps ...

cheers, MuS

View solution in original post

MuS
Legend

Hi premg,

Basically you set a default destination for your data in transforms.conf and/or in outputs.conf. If you want to manage default destinations in one .conf file, do it in transforms.conf, because _TCP_ROUTING from transforms.conf will overwrite the outputs.conf defaultGroup stanza.

hope this helps ...

cheers, MuS

MuS
Legend

update ping

0 Karma

MuS
Legend

ok, now I got your point. There is no link and _TCP_ROUTING from transforms.conf will over write defaultGroup in outputs.conf. If you want to manage default groups in one single file, stick to the transforms.conf and configure everything in there.

0 Karma

premg
Engager

Many thanks MuS.
I'll leave [routeAllEvents].
However, how do Splunk pick up default group in [routeFilteredEvents] stanza?

0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...