How do I redirect data to a different set of indexers via my heavy forwarder?

[daniel333]

All,

I want to set aside a handful of indexers to store important data. I have a heavy forwarder setup. So should be an option in transforms.conf to redirect specific sources.. But for the life of me skimming the docs isn't working.

thanks,

[pradeepkumarg · Accepted solution]

You can define a separate group of indexers in your outputs.conf and target that group for each source you wish in the inputs.conf by setting below property

_TCP_ROUTING = tcpout_group_name

[daniel333]

Hey,

So what I didn't get is you still need an outputs.conf for this. Here is my final product.

Placed this into one app - 

# props.conf
[host::*myserver*]
TRANSFORMS-routing=pciRouting

# transforms.conf
[pciRouting]
REGEX=.
DEST_KEY=_TCP_ROUTING
FORMAT=pci-group

# outputs.conf
[tcpout]

[tcpout:pci-group]
server = myindexer.mydomain.com:9997

[pradeepkumarg]

You do not need any transforms. You Just need changes in outputs.conf and inputs.conf. Examples below

outputs.conf

[tcpout]

defaultGroup = Regular_Indexers

[tcpout:Regular_Indexers]
forceTimebasedAutoLB = true
autoLB = true
autoLBFrequency = 10
server = indexer1:9997,indexer2:9997,indexer3:9997,indexer4:9997,indexer5:9997
compressed = true

[tcpout:SPECIAL_Indexers]
forceTimebasedAutoLB = true
autoLB = true
autoLBFrequency = 10
server = indexer6:9997,indexer7:9997
compressed = true

inputs.conf

[monitor:///your/path/to/log_file.log]

_TCP_ROUTING = SPECIAL_Indexers //This will route the data for this source to the SPECIAL indexers
sourcetype = my_sourcetype1
index=my_index1

[monitor:///your/path/to/second_log_file.log]
sourcetype = my_sourcetype2 //This will go to the default group (Regular indexers)
index=my_index1