Getting Data In

heavy forwarder to route base on _raw values

gooza
Communicator

I'm trying to use the heavy forwarder to route data to different indexes based on values in _raw , is this possible ?

the configuration files are:

inputs.conf:

[tcp://9997]

sourcetype = FromFooandBarbysinglehost

props.conf:

[FromFooandBarbysinglehost]

BREAK_ONLY_BEFORE = ^

TRANSFORMS-routing = FromFoo,FromBar

transforms.conf:

[FromFoo]

REGEX = (?i) From|.+?Foo

DEST_KEY = _TCP_ROUTING

FORMAT = outtoFoo

[FromBar]

REGEX = (?i) From|.+?Bar

DEST_KEY = _TCP_ROUTING

FORMAT = outtoBar

outputs.conf:

[tcpout:outtoFoo]

server = 10.10.10.10:1111

sendCookedData = false

[tcpout:outtoBar]

server = 10.10.10.10:2222

sendCookedData = false

in the indexer 10.10.10.10 the TCP port 1111 is indexed to foo index , and 2222 is indexed to Bar index)

me problem is that I see both foo data and bar data in both indexes , it is like there is no termination to the transforms process and both are sent to both ports.

I double checked my REGEX in the search bar in splunk and it does show only the relevant data

what am I missing ?

0 Karma

Jason
Motivator

_TCP_ROUTING is used to send data to another indexer. If the data is already at the indexer you want, just use the _MetaData:Index key to change the index for that event.

[FromFoo]
REGEX = (?i) From|.+?Foo
DEST_KEY = _MetaData:Index
FORMAT = foo_index

[FromBar]
REGEX = (?i) From|.+?Bar
DEST_KEY = _MetaData:Index
FORMAT = bar_index
0 Karma

gooza
Communicator

no , the tcpout redirect the data to different TCP ports on the indexer and each port has its own source type and index

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

when your data arrives in the indexers, what is the sourcetype? Is it FromFooandBarbysinglehost?

0 Karma
Get Updates on the Splunk Community!

More Ways To Control Your Costs With Archived Metrics | Register for Tech Talk

Tuesday, May 14, 2024  |  11AM PT / 2PM ET Register to Attend Join us for this Tech Talk and learn how to ...

.conf24 | Personalize your .conf experience with Learning Paths!

Personalize your .conf24 Experience Learning paths allow you to level up your skill sets and dive deeper ...

Threat Hunting Unlocked: How to Uplevel Your Threat Hunting With the PEAK Framework ...

WATCH NOWAs AI starts tackling low level alerts, it's more critical than ever to uplevel your threat hunting ...