How to send syslog data to the indexer and another TCP listener? (Part 3)

[Log_wrangler]

Hi All,

Thank you for the assistance so far.

I just want to confirm my understanding and ask a follow-up REGEX question in regards to [routeAll] and [routeSubset].

So if I edit the following:
Edit $SPLUNK_HOME/etc/system/local/outputs.conf

[tcpout]
defaultGroup=nothing <----- setting defaultGroup to "nothing" defines that default group is receiving nothing?

As directed with the following stanzas (below):
"everything" (old and new source feeds) goes to the indexer(s)
"subsidiary" goes only to 3rd party TCP receiver....

[tcpout:Everything]
disabled=false
server=x.x.x.x:9997

[tcpout:Subsidiary]
disabled=false
sendCookedData=false
server=x.x.x.x:1234

Is that correct ???

Question:
I am in the staging phase, so I have not had a chance to test a regex for the following:

Please advise how to write the REGEX for the [routeAll] to send all data to indexer(s); and how to write the REGEX for the [routeSubset] to only send uncooked data to the 3rd party TCP receiver.

I need an example if possible.,

For example, REGEX=(SYSTEM|CONFIG|THREAT), how did the author determine this is the correct expression???

[routeAll]
REGEX=(.)<--------- This is where I would specify all data would continue to the indexer(s)?
DEST_KEY=_TCP_ROUTING
FORMAT=Everything

[routeSubset]
REGEX=(SYSTEM|CONFIG|THREAT) <--------- This is where I would specify which data would go to the 3rd party app?
DEST_KEY=_TCP_ROUTING
FORMAT=Subsidiary

Any documentation on this is appreciated.

Thank you

[harsmarvania57 · Accepted solution]

Hi @Log_wrangler,

If you want to send all data (Assuming multiple sourcetypes) are sending data to HF and HF to IDX then you can do following configuration.

outputs.conf

[tcpout]
defaultGroup=Everything

[tcpout:Everything]
disabled=false
server=x.x.x.x:9997

[tcpout:Subsidiary]
disabled=false
sendCookedData=false
server=x.x.x.x:1234

props.conf for only those sourcetype for which you want to transfer data to 3rd party server.

[yoursourcetype]
TRANSFORMS-routesubset = routesubsetdata

transforms.conf, in below configuration if your event contains SYSTEMS OR CONFIG OR THREAT word then those event will route to 3rd party but those events goes to Indexers as well due to defaultGroup=Everything. So based on your requirement you can create regex or if you need help then you can ask new question with some sample data for regex.

[routesubsetdata]
REGEX=(SYSTEM|CONFIG|THREAT)
DEST_KEY=_TCP_ROUTING
FORMAT=Subsidiary

I hope this helps.

Thanks,
Harshil

[Log_wrangler]

Thank you for the clarification.

What happens if the following is in place?

[tcpout]
defaultGroup=nothing

[harsmarvania57]

If you give defaultGroup=nothing in that case you need to configure props.conf and transforms.conf to route those data to Indexer and 3rd party system as given by you in question. But you need to do same configuration for all sourcetypes otherwise sourcetypes which are not configured to routeall data to Indexers will be dropped on HF.

[Log_wrangler]

Thank you, that is what I was concerned about....