Getting Data In

Using Splunk heavy forwarder - Filter data before TCP routing - What's wrong with my configuration?

patouellet
Path Finder

Hi,

I'm using a Splunk Heavy Forwarder with props.conf, transforms.conf and outputs.conf to selectively send events to different splunk Indexers based on the sourcetype. That works well.

But now I have a new need: For a specific sourcetype (csi_pclog), I need to get rid of most events first, then route remaining events to a specific indexer. For some reason, I'm having trouble making this work. I'm not receiving any events and I've checked in the actual file I'm forwarding - there's data in there.

Basically all events for sourcetype=csi_pclog need to be dropped, except events with the word "Login". Here's what I have:

props.conf

[csi_pclog]
TRANSFORMS-routing=itseccsiappRouting, itseccsiappDiscard

transforms.conf

[itseccsiappDiscard]
REGEX=.
DEST_KEY= queue
FORMAT=nullQueue

[itseccsiappRouting]
REGEX=Login
DEST_KEY=_TCP_ROUTING
FORMAT=default-autolb-group

outputs.conf

[tcpout]
defaultGroup = default-autolb-group
forwardedindex.3.whitelist = (_audit|_internal|_introspection|_telemetry)

[tcpout:default-autolb-group]
server = indexer1:9997

[tcpout-server://indexer1:9997]

[tcpout:itsechf]
server = indexer2:9997

What am I doing wrong?

0 Karma
1 Solution

gjanders
SplunkTrust
SplunkTrust

If you refer to route and filter data you probably want to do (quoting from the documentation):

(1) Edit props.conf and add the following:

[source::/var/log/messages]
TRANSFORMS-set= setnull,setparsing

(2) Edit transforms.conf and add the following:
[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = \[sshd\]
DEST_KEY = queue
FORMAT = indexQueue

(3) Restart Splunk Enterprise.

In other words you want to change the queue not the routing when you want to include the data you previously sent to the null queue...

Right now you do appear to be dropping everything to the null queue so no data will be sent at all.

View solution in original post

0 Karma

patouellet
Path Finder

garethatiag answer was key to figuring this out. The full working config is:

props.conf

[csi_pclog]
TRANSFORMS-routing=itseccsiappDiscard,itseccsiappKeep, itseccsiappRouting

transforms.conf

[itseccsiappDiscard]
REGEX= .
DEST_KEY= queue
FORMAT=nullQueue

[itseccsiappKeep]
REGEX=Login
DEST_KEY=queue
FORMAT=indexQueue

[itseccsiappRouting]
REGEX= .
DEST_KEY=_TCP_ROUTING
FORMAT=itsechf

gjanders
SplunkTrust
SplunkTrust

If you refer to route and filter data you probably want to do (quoting from the documentation):

(1) Edit props.conf and add the following:

[source::/var/log/messages]
TRANSFORMS-set= setnull,setparsing

(2) Edit transforms.conf and add the following:
[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = \[sshd\]
DEST_KEY = queue
FORMAT = indexQueue

(3) Restart Splunk Enterprise.

In other words you want to change the queue not the routing when you want to include the data you previously sent to the null queue...

Right now you do appear to be dropping everything to the null queue so no data will be sent at all.

0 Karma

patouellet
Path Finder

Thanks for the reply. I kinda get what you are saying, I think. But I'm not indexing on this HF, I want to send it to a specific tcpout group.

Let's say I do what you suggest, the events will go to the default output group, right? How to I tell it to route to a different group?

0 Karma

gjanders
SplunkTrust
SplunkTrust

There is likely more than one way to do this but why not specify in your inputs.conf the _TCP_ROUTING = setting? This way you can send the data to the correct TCP output.

Since the transforms.conf you have will drop everything to the null queue by default, you can use your second regular expression (Login) to ensure that data gets forwarded (and since the inputs.conf has already specified which TCP output queue you are using you won't need to override in the transforms.conf).

In terms of config:
props.conf

[csi_pclog]
TRANSFORMS-routing=itseccsiappDiscard,itseccsiappRouting

transforms.conf

[itseccsiappDiscard]
REGEX=.
DEST_KEY= queue
FORMAT=nullQueue

[itseccsiappRouting]
REGEX=Login
DEST_KEY=queue
FORMAT=indexQueue

In your inputs.conf

[<yourstanza>]
_TCP_ROUTING = itsechf
0 Karma

gjanders
SplunkTrust
SplunkTrust

Or do multiple transforms to change the routing and the discard/include data if you prefer...

0 Karma

patouellet
Path Finder

Think I would prefer the multiple transforms option. Care to help again? Never done it, unsure what do do.

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

Try to reverse order in props.conf

[csi_pclog]
TRANSFORMS-routing= itseccsiappDiscard,itseccsiappRouting

And then restart Splunk on HF

0 Karma

patouellet
Path Finder

Thanks for the reply. Tried it, no change. Still not getting events. I see the connection being established between the client and the HF, so it has to be my props/transforms/outputs combination that aren't quite right.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...