Splunk Search

How to configure forwarder to send different information to 2 different indexers

DyJohnnY
Explorer

Hi,

We now have a setup in which we use splunk like this. Forwarders deployed on windows Domain Controllers, that receive every log except success audit events.

Success audit events are dumped using props.conf files and transforms.conf files. configuration is below.

props.conf

[WinEventLog:Security]
TRANSFORMS-set = Dump_Success_Audit 



transforms.conf

[WinEventLog:Security]
TRANSFORMS-set = Dump_Success_Audit


[Dump_Success_Audit]
REGEX = (?m)(?i)^Type=(Success Audit)
DEST_KEY = queue
FORMAT = nullQueue





outputs.conf
[tcpout]
defaultGroup = splunk_5514
disabled = false

[indexAndForward]
index = false

[tcpout:splunk_5514]
server=X.X.X.X:5514
heartbeatFrequency=45
maxQueueSize=100500

What we want is following:

keep the current configuration (or its results) but also capture some "Success Audit Events" (we will do the filtering based on event ids) and send just those events to another splunk instance.

Does anyone know how we can approach this problem?

Thanks for the help.

2 Solutions

Stephen_Sorkin
Splunk Employee
Splunk Employee

This is a relatively straightforward use of the _TCP_ROUTING key of index-time events.

First, add to outputs.conf:

[tcpout:splunk_success_audit]
server=X.X.X.X:5514
heartbeatFrequency=45
maxQueueSize=100

Next in transforms.conf:

[Dump_Success_Audit]
REGEX = (?m)(?i)^Type=(Success Audit)
DEST_KEY = _TCP_ROUTING
FORMAT = splunk_success_audit

This will forward ALL Type=Success Audit to the other system. To be more selective here, you can set up another regex to route the undesired events to the nullQueue. Just create a copy of Dump_Success_Audit, say Dump_Success_Audit_2 and have that run from props.conf: TRANSFORMS-set = Dump_Success_Audit Dump_Success_Audit_2.

As an aside, you shouldn't set your maxQueueSize to more than 1000. I usually suggest 100 on LWF and 1000 on standard forwarders. This will result in the lowest latency and memory usage on the forwarder.

View solution in original post

0 Karma

ftk
Motivator

You should be able to add some routing entries to take care of this. Check out Route and Filter to Target Groups in the docs.

Basically you can add a

[WinEventLog:Security]
TRANSFORMS-routing = routeToFoo

entry to props.conf with a configuration such as the following in transforms.conf:

[routeToFoo]
REGEX=(?m)(?i)^EventCode=(540|542|544)
DEST_KEY=_TCP_ROUTING
FORMAT=fooGroup

and then add the routing group in your outputs.conf:

[tcpout:fooGroup]
server=10.1.1.1:9997

Stephen_Sorkin
Splunk Employee
Splunk Employee

This is a relatively straightforward use of the _TCP_ROUTING key of index-time events.

First, add to outputs.conf:

[tcpout:splunk_success_audit]
server=X.X.X.X:5514
heartbeatFrequency=45
maxQueueSize=100

Next in transforms.conf:

[Dump_Success_Audit]
REGEX = (?m)(?i)^Type=(Success Audit)
DEST_KEY = _TCP_ROUTING
FORMAT = splunk_success_audit

This will forward ALL Type=Success Audit to the other system. To be more selective here, you can set up another regex to route the undesired events to the nullQueue. Just create a copy of Dump_Success_Audit, say Dump_Success_Audit_2 and have that run from props.conf: TRANSFORMS-set = Dump_Success_Audit Dump_Success_Audit_2.

As an aside, you shouldn't set your maxQueueSize to more than 1000. I usually suggest 100 on LWF and 1000 on standard forwarders. This will result in the lowest latency and memory usage on the forwarder.

0 Karma

DyJohnnY
Explorer

Hi guys,

Thanks a lot for the feedback, in the end we kind of figured it out how to do it, based on the basic splunk examples on the documentation and your feedback here.

Question can be closed now, hope it will help someone else aswell

0 Karma

jfraiberg
Communicator
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...