Getting Data In

How to route locally indexed events (from the perspective of an indexer) to another environment?

w531t4
Path Finder

All --

I'm seeking any advice I can get at this point. A little background. I manage two different user communities (A and B). Community A consists of a universal forwarder aggregator machine, as well as a combination indexer/search head. I believe the topology of community B to be irrelevant.

I've been given the task of taking a subset of events from community A and sending them to community B. To further complicate things, the data that I'm seeking to send to community B is produced by a saved search (using |output csv ) by the single indexer present in community A. The CSV is output to the local filesystem of community A's indexer, and an inputs.conf file monitors that CSV and imports the contents in to a index dedicated to housing only these events.

My attempts as of this point have consisted of specifying the following in community A's indexer outputs.conf file

[tcpout]
indexAndForward = true
forwardedindex.0.whitelist = indexforthisspecificpurpose
forwardedindex.1.blacklist = _.*
forwardedindex.2.blacklist = .*

[tcpout:mytarget]
server=blahip:9997
... ssl config ...

Community A indexer/search head receives a feed of events from community A via a universal forwarder aggregator. These events should all be written and stored to the indexer.

Given this config, I'm not observing the intended result (events only destined to index=indexforthisspecificpurpose). I'm seeing results go to index=os (unix app) and one of our summary indexes. Knowing this, I can say that as far as the link between communities, the connection works. However, the whitelisting/selection of data is not behaving as desired.

Has anyone tried to do something like this? Do you have any suggestions how to accomplish this?

Thanks!

0 Karma
1 Solution

muebel
SplunkTrust
SplunkTrust

Hi w531t4, given the peculiar way that the index and forward whitelisting mechanism works, I believe you will have success by doing something like:

forwardedindex.0.blacklist =
forwardedindex.0.whitelist =
forwardedindex.1.blacklist =
forwardedindex.1.whitelist =
forwardedindex.2.whitelist =
forwardedindex.2.blacklist =
forwardedindex.0.blacklist = .*
forwardedindex.1.blacklist = _.*
forwardedindex.2.whitelist = indexforthisspecificpurpose

These settings null-out the filters and then reset them. Try this out and let me know if it helps.

View solution in original post

0 Karma

muebel
SplunkTrust
SplunkTrust

Hi w531t4, given the peculiar way that the index and forward whitelisting mechanism works, I believe you will have success by doing something like:

forwardedindex.0.blacklist =
forwardedindex.0.whitelist =
forwardedindex.1.blacklist =
forwardedindex.1.whitelist =
forwardedindex.2.whitelist =
forwardedindex.2.blacklist =
forwardedindex.0.blacklist = .*
forwardedindex.1.blacklist = _.*
forwardedindex.2.whitelist = indexforthisspecificpurpose

These settings null-out the filters and then reset them. Try this out and let me know if it helps.

0 Karma

w531t4
Path Finder

from outputs.conf

 * The filters can start from either whitelist or blacklist. They are tested from forwardedindex.0 to forwardedindex.<max>.
 *** If both forwardedindex.<n>.whitelist and forwardedindex.<n>.blacklist are present for the same value of n, then
   forwardedindex.<n>.whitelist is honored. forwardedindex.<n>.blacklist is ignored in this case.**

after your suggestion, i still receive all locally generated events.

0 Karma

muebel
SplunkTrust
SplunkTrust

ah, try

forwardedindex.2.whitelist = indexforthisspecificpurpose
0 Karma

w531t4
Path Finder

this --- this worked. Thank you Thank you Thank you. I would have never come up with this just by looking at the outputs.conf.spec file.

muebel
SplunkTrust
SplunkTrust

glad to help 😄

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...