Getting Data In

how to index locally and forward a specific sourcetype

sylbaea
Communicator

Hello,

When events with a specific sourcetype arrive on my indexers, I would like to have both local indexing (default for any kind of sourcetype) but also forward them to another Splunk indexer.

So far I got this... It does properly forward this sourcetype to the external indexer.
But no longer index the events locally.

outputs.conf
[tcpout:externalIndexer]
server = external_indexer:9997

props.conf
[SourceTypeToForward]
TRANSFORMS-routing = sendToExternalIndexer

transforms.conf
[sendToExternalIndexer]
REGEX = .
DEST_KEY = _TCP_ROUTING
FORMAT = externalIndexer

How can I enhance this config to have both ?

0 Karma
1 Solution

gcusello
SplunkTrust
SplunkTrust

Hi sylbaea,
see at http://docs.splunk.com/Documentation/Splunk/latest/Forwarding/Routeandfilterdatad the part "Perform selective indexing and forwarding"
in details for locally index logs and forward a part of them you have to:
In outputs.conf, add the [indexAndForward] stanza:

[indexAndForward]
index=true
selectiveIndexing=true 

Note: This is a global stanza, and only needs to appear once in outputs.conf.
Include the target group stanzas for each set of receiving indexers:

[tcpout:<target_group>]
server = <ip address>:<port>, <ip address>:<port>, ...
... 

The forwarder uses the named in inputs.conf to route the inputs.

In inputs.conf, add the _INDEX_AND_FORWARD_ROUTING setting to the stanzas of each input that you want to index locally:

[input_stanza]
_INDEX_AND_FORWARD_ROUTING=<any_string>
...

Add the _TCP_ROUTING setting to the stanzas of each input that you want to forward:

[input_stanza]
_TCP_ROUTING=<target_group>
...

The is the name used in outputs.conf to specify the target group of receiving indexers.

Bye.
Giuseppe

0 Karma

sylbaea
Communicator

Thanks. I had a look already to this but it is unclear to me how it could apply to my case.

0 Karma

jbarlow_splunk
Splunk Employee
Splunk Employee

It has an example as well..

Perform selective indexing and forwarding

With a heavy forwarder only, you can index and store data locally, as well as

forward the data onwards to a receiving indexer. There are two ways to do

this:

1. In outputs.conf:

[tcpout]
defaultGroup = indexers

[indexAndForward]
index=true
selectiveIndexing=true

[tcpout:indexers]
server = 10.1.1.197:9997, 10.1.1.200:9997

2. In inputs.conf, Add _INDEX_AND_FORWARD_ROUTING for any data that you want

index locally, and

_TCP_ROUTING= for data to be forwarded.

[monitor:///var/log/messages/]
_INDEX_AND_FORWARD_ROUTING=local

[monitor:///var/log/httpd/]
_TCP_ROUTING=indexers

0 Karma

sylbaea
Communicator

I got your point. Thanks 🙂
Will test

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...