Getting Data In

How can I forward only _internal index data from indexer to the new indexer?

benazir
Explorer

I am facing a problem in forwarding the _internal data to the new indexer.

my case is I have to forward only _internal data from all the indexers to new indexer servers because in our environment we have dedicated indexer for _internal data.

when i do this below entry in one of the indexer
inputs.conf:

[monitor:///opt/splunk/idx/splunk/var/log/splunk]
_TCP_ROUTING = management

outputs.conf

[tcpout]
forwardedindex.0.blacklist = .*
forwardedindex.1.whitelist = _internal
forwardedindex.2.whitelist = _audit
forwardedindex.filter.disable = false
disabled=false

[tcpout:management]
server = 10.178.48.66:9997

This makes all the data to forward from this particular indexer to the new indexer, I need only _internal data to get forwarded.

I tried using props.conf and transforms.conf too. It's not working. I don't want to store the _internal data in this indexer, it should present only in the new indexers.
Kindly need your help.

0 Karma

somesoni2
SplunkTrust
SplunkTrust

Try with this outputs.conf (should be etc/apps under some_app/local OR last resort, under etc/system/local)

[tcpout]
indexAndForward = true

[tcpout:management]
server = 10.178.48.66:9997

[indexAndForward]
index=true
0 Karma

benazir
Explorer

I tried this option, what it does it , it keeps a copy of internal logs here in the old indexers and forward to new indexers too.

but my case is , I need to see the _internal data of that particular indexers only in the new indexers, not on the source indexer, when I search data from search head for _internal index..
since we have dedicated search heads , for different cluster of indexers.

Kindly need to your advice, how to just forward, without doing local indexing .

0 Karma

benazir
Explorer

I haven given the outputs.conf file like below :
[tcpout]
forwardedindex.0.blacklist = .*
forwardedindex.1.whitelist = _internal
forwardedindex.2.whitelist = _audit
forwardedindex.filter.disable = false
disabled=false
indexAndForward = true

[tcpout:management]
server = 10.178.48.66:9997

[indexAndForward]
index = true

Now this is how it works, I cant find any other data forwarded to new management indexer ( that's good)
but the problem is _internal data is routed to main index in the new server - 10.178.48.66 and missing few logs like splunkd,metrics all.
meantime in the old indexer I am still seeing the data from main as well as _internal indexes.

0 Karma

deepashri_123
Motivator

Is there any reason why a particular indexer set for internal indexes only? This is not the best practice to do so.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...