Getting Data In

How to use indexer master as listener for data, but actual data will be stored on slaves?

Rialf1959
Explorer

Hello,
How to use indexer master as listener for data, but actual data will be stored on slaves?
Is possible to forward data from universal forwarders to indexer master and have random number of indexer slaves?
With this config - data are stored in indexer master. Where is a problem? Thanks

version: '3.5'
services:
  master:
    hostname: master
    image: splunk/splunk:7.0.2
    ports:
      - 8001:8000
      - 9997:9997
    environment:
     - "SPLUNK_ENABLE_LISTEN=9997"
     - "SPLUNK_START_ARGS=--accept-license --answer-yes "
     - "SPLUNK_CMD_1=edit user admin -password admin -role admin -auth admin:changeme"
     - "SPLUNK_CMD_2=edit cluster-config -mode master -replication_factor 1 -search_factor 1 -secret mytoken -cluster_label mycluster"
     - "SPLUNK_CMD_3=set servername splunk-master.example.com"
     - "SPLUNK_CMD_4=set default-hostname splunk-master.example.com"
     - "SPLUNK_CMD_5=restart"


  head:
    hostname: head
    image: splunk/splunk:7.0.2
    ports:
      - 8002:8000
    environment:
      - "SPLUNK_START_ARGS=--accept-license --answer-yes "
      - "SPLUNK_CMD_1=edit user admin -password admin -role admin -auth admin:changeme"
      - "SPLUNK_CMD_2=edit cluster-config -mode searchhead -master_uri https://master:8089 -replication_port 9887 -secret mytoken"
      - "SPLUNK_CMD_3=set servername splunk-head.example.com"
      - "SPLUNK_CMD_4=set default-hostname splunk-head.example.com"
      - "SPLUNK_CMD_5=restart"

  slave:
    hostname: slave
    image: splunk/splunk:7.0.2
    ports:
      - 8003:8000
    environment:
      - "SPLUNK_START_ARGS=--accept-license --answer-yes "
      - "SPLUNK_CMD_1=edit user admin -password admin -role admin -auth admin:changeme"
      - "SPLUNK_CMD_2=edit cluster-config -mode slave -master_uri https://master:8089 -replication_port 9887 -secret mytoken"
      - "SPLUNK_CMD_3=set servername splunk-slave.example.com"
      - "SPLUNK_CMD_4=set default-hostname splunk-slave.example.com"
      - "SPLUNK_CMD_5=restart"
Tags (2)
0 Karma

tiagofbmm
Influencer

Are you forwarding the data from the Cluster Master to the Indexers?

In the cluster master

1 - Create an outputs.conf pointing to the indexer:

https://docs.splunk.com/Documentation/Forwarder/7.0.2/Forwarder/Configureforwardingwithoutputs.conf

2 - Create an app in the $SPLUNK_HOME/etc/master-apps and put an inputs.conf there with the stanza:

[splunktcp:<port_you_are_sending_from_the_Cluster_master>]

Then in the Cluster Master

$SPLUNK_HOME/bin/splunk validate cluster-bundle
$SPLUNK_HOME/bin/splunk apply cluster-bundle 

And with this you will be forwarding all the data arriving to the Cluster Master to the indexer layer like you want

0 Karma

richgalloway
SplunkTrust
SplunkTrust

This makes the Master a choke point and SPOF. Not a good idea, IMO.

---
If this reply helps you, Karma would be appreciated.
0 Karma

tiagofbmm
Influencer

Agreed, not a good idea indeed. But if for some reason the user is into using the CM as a Listener, I believe this would be the way

0 Karma

richgalloway
SplunkTrust
SplunkTrust

The Master Node is not a data forwarder. That is, you can't send data to it and expect that data to land on a search peer.

It looks like you want Indexer Discovery. This feature tells the forwarders to ask the Master Node for a list of indexers. The forwarders then use that list to determine where to send data.
See http://docs.splunk.com/Documentation/Splunk/7.0.2/Indexer/indexerdiscovery for details.

---
If this reply helps you, Karma would be appreciated.
0 Karma

Rialf1959
Explorer

This is not working in Docker because if a container is not running with --network host,
master node will forward internal ip address, not public one -> is not accessible.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...