Hello,
How to use indexer master as listener for data, but actual data will be stored on slaves?
Is possible to forward data from universal forwarders to indexer master and have random number of indexer slaves?
With this config - data are stored in indexer master. Where is a problem? Thanks
version: '3.5'
services:
master:
hostname: master
image: splunk/splunk:7.0.2
ports:
- 8001:8000
- 9997:9997
environment:
- "SPLUNK_ENABLE_LISTEN=9997"
- "SPLUNK_START_ARGS=--accept-license --answer-yes "
- "SPLUNK_CMD_1=edit user admin -password admin -role admin -auth admin:changeme"
- "SPLUNK_CMD_2=edit cluster-config -mode master -replication_factor 1 -search_factor 1 -secret mytoken -cluster_label mycluster"
- "SPLUNK_CMD_3=set servername splunk-master.example.com"
- "SPLUNK_CMD_4=set default-hostname splunk-master.example.com"
- "SPLUNK_CMD_5=restart"
head:
hostname: head
image: splunk/splunk:7.0.2
ports:
- 8002:8000
environment:
- "SPLUNK_START_ARGS=--accept-license --answer-yes "
- "SPLUNK_CMD_1=edit user admin -password admin -role admin -auth admin:changeme"
- "SPLUNK_CMD_2=edit cluster-config -mode searchhead -master_uri https://master:8089 -replication_port 9887 -secret mytoken"
- "SPLUNK_CMD_3=set servername splunk-head.example.com"
- "SPLUNK_CMD_4=set default-hostname splunk-head.example.com"
- "SPLUNK_CMD_5=restart"
slave:
hostname: slave
image: splunk/splunk:7.0.2
ports:
- 8003:8000
environment:
- "SPLUNK_START_ARGS=--accept-license --answer-yes "
- "SPLUNK_CMD_1=edit user admin -password admin -role admin -auth admin:changeme"
- "SPLUNK_CMD_2=edit cluster-config -mode slave -master_uri https://master:8089 -replication_port 9887 -secret mytoken"
- "SPLUNK_CMD_3=set servername splunk-slave.example.com"
- "SPLUNK_CMD_4=set default-hostname splunk-slave.example.com"
- "SPLUNK_CMD_5=restart"
Are you forwarding the data from the Cluster Master to the Indexers?
In the cluster master
1 - Create an outputs.conf pointing to the indexer:
https://docs.splunk.com/Documentation/Forwarder/7.0.2/Forwarder/Configureforwardingwithoutputs.conf
2 - Create an app in the $SPLUNK_HOME/etc/master-apps and put an inputs.conf there with the stanza:
[splunktcp:<port_you_are_sending_from_the_Cluster_master>]
Then in the Cluster Master
$SPLUNK_HOME/bin/splunk validate cluster-bundle
$SPLUNK_HOME/bin/splunk apply cluster-bundle
And with this you will be forwarding all the data arriving to the Cluster Master to the indexer layer like you want
This makes the Master a choke point and SPOF. Not a good idea, IMO.
Agreed, not a good idea indeed. But if for some reason the user is into using the CM as a Listener, I believe this would be the way
The Master Node is not a data forwarder. That is, you can't send data to it and expect that data to land on a search peer.
It looks like you want Indexer Discovery. This feature tells the forwarders to ask the Master Node for a list of indexers. The forwarders then use that list to determine where to send data.
See http://docs.splunk.com/Documentation/Splunk/7.0.2/Indexer/indexerdiscovery for details.
This is not working in Docker because if a container is not running with --network host,
master node will forward internal ip address, not public one -> is not accessible.