Deployment Architecture

How to forward ONLY _internal logs from standalone Search Head to Indexer Cluster?

naqviah
Explorer

Can someone guide me on how to forward ONLY _internal logs from a standalone SH (Search Head) to an indexer cluster in a distributed environment?

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi naqviah,
you have to configure your Search Head as an Heavy Forwarder, using web interface:

  • [Settings -- Forwarding and receiving -- Forwarding defaults] to say to the Search Head to not locally index logs;
  • [Settings -- Forwarding and receiving -- Configure forwarding] to say to the Search Head which are Indexers to send logs.

and restart Search Head.

Bye.
Giuseppe

0 Karma

jonmargulies
Path Finder

What you want to do is create a local outputs.conf configuration on the Search Head that looks like this:

  [tcpout]
  forwardedindex.0.whitelist = .*
  forwardedindex.1.blacklist = _.*
  forwardedindex.2.whitelist = (_internal)

This will replace the default outputs.conf entry, which sends more internal indexes to indexers. Other than the above, all you'll have to do is set outputs.conf to point at the indexer cluster.

See this old answer for more detail on how the forwardedindex whitelists and blacklists work: https://answers.splunk.com/answers/339930/how-do-forwardedindex-whitelists-and-blacklists-wo.html

0 Karma

naqviah
Explorer

Do i have to configure the inputs.conf on the IDX clusters?

This is what i currently have, but there is a communication error between the SH and the IDX_Cluster:

[indexAndForward]
index = true

[tcpout]
defaultGroup = idx-indexers
forwardedindex.filter.disable = false
indexAndForward = 1

[tcpout]
forwardedindex.0.whitelist = .*
forwardedindex.1.blacklist = _.*
forwardedindex.2.whitelist = (_internal)

[tcpout:idx-indexers]
autoLBFrequency = 40
disabled = 0
server = :9997,:9997:9997
sslCertPath = $SPLUNK_HOME/etc/auth/server.pem
sslPassword =
sslRootCAPath = $SPLUNK_HOME/etc/auth/cacert.pem
sslVerifyServerCert = 0
useACK = 1

Anything im doing wrong here?

0 Karma

brreeves_splunk
Splunk Employee
Splunk Employee

The SH should only be generating internal logs anyway...unless you want to skip the other internal indexes.

What EXACTLY are you trying to accomplish?

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...