Deployment Architecture

Can I use indexer discovery to forward search head cluster data to multi-site indexer cluster?

basanthp
Path Finder

We have a setup of 6 search head clusters connected to 2 multi-site indexer clusters, using Splunk v6.3.1. My objective is to forward search head cluster data to multi-site indexer cluster.

As per following documentation, I understand this can be achieved using outputs.conf
http://docs.splunk.com/Documentation/Splunk/6.3.1/DistSearch/Forwardsearchheaddata
tcpoutput stanza (server=host1, host2, host3)

But I want the cluster master node to handle this (similar to indexer discovery in forwarders).

q1. Is this achievable?
q2. Will the data be replicated if it is sent to a set of indexers in multi-site indexers?

0 Karma
1 Solution

basanthp
Path Finder

Yes. Technically possible. We need to add the role of forwarder to search head. We also need to disable indexing on the SH so data is not duplicated according to the following section.

“2. Configure the search head as a forwarder. Create an outputs.conf file on the search head that configures the search head for load-balanced forwarding across the set of search peers (indexers). You must also turn off indexing on the search head, so that the search head does not both retain the data locally as well as forward it to the search peers.”

Since the SH now is also a forwarder, we should be able to use the outputs.conf like, on any other forwarder, so indexerDiscovery should work the same. Forwarder contacts the cluster master and gets the list of indexers.

We have tested this in our LAB and implemented in PROD.

View solution in original post

0 Karma

basanthp
Path Finder

Yes. Technically possible. We need to add the role of forwarder to search head. We also need to disable indexing on the SH so data is not duplicated according to the following section.

“2. Configure the search head as a forwarder. Create an outputs.conf file on the search head that configures the search head for load-balanced forwarding across the set of search peers (indexers). You must also turn off indexing on the search head, so that the search head does not both retain the data locally as well as forward it to the search peers.”

Since the SH now is also a forwarder, we should be able to use the outputs.conf like, on any other forwarder, so indexerDiscovery should work the same. Forwarder contacts the cluster master and gets the list of indexers.

We have tested this in our LAB and implemented in PROD.

0 Karma

sylvainlectra
Explorer

Hi,

I've copied my outputs.conf from my other forwarders on my search head in /opt/splunk/etc/system/local/outputs.conf :

[indexer_discovery:master1]
pass4SymmKey = ************
master_uri = https://*********:8089

[tcpout:group1]
autoLBFrequency = 30
forceTimebasedAutoLB = true
indexerDiscovery = master1
useACK = true

[tcpout]
defaultGroup = group1

After a restart I'm able to add monitors to the indexes present on the peers which seems to mean that discovery occured but I see no data from the heads when I do a search and "list forward-server" on the search heads returns nothing

root@********-spk-search1:/opt/splunk/etc# /opt/splunk/bin/splunk list forward-server
Active forwards:
        None
Configured but inactive forwards:
        None

Did I miss something ?

Also how do I disable local indexing on the search heads ?

Regards.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...