Deployment Architecture

Waiting for requisite number of peers to join the cluster - https://127.0.0.1:8089. Cluster has only 0 peers (waiting for 3 peers to join the cluster)

ashishlal82
Explorer

I am receiving this error on my CM. The peer nodes are not able to join CM

0 Karma

micahkemp
Champion

What is preventing them from joining the cluster? Are there network or host based firewalls blocking the traffic? Are the peer nodes properly configured?

We need more information to help.

0 Karma

ashishlal82
Explorer

On the indexer boxes I am receving 1069 error that splunkweb services are not running. You think that could be the issue?

0 Karma

micahkemp
Champion

splunkweb doesn't need to be running on the indexers, so that isn't necessarily pointing to the issue. Is splunkd running on the indexers?

0 Karma

ashishlal82
Explorer

yes Splunkd is running on the indexers. But splunk webservice is not.

0 Karma

micahkemp
Champion

Are the indexers configured to be a part of the cluster? What do you get for this command when run on an indexer:

splunk btool server list clustering --debug
0 Karma

ashishlal82
Explorer

[general]
serverName = Server-name
pass4SymmKey = $1$tVLM068URVKj

[queue]
maxSize = 0

[queue=indexQueue]
maxSize = 1GB

[queue=parsingQueue]
maxSize = 100MB

[queue=aggQueue]
maxSize = 100MB

SHOULD_LINEMERGE = false

[queue=auditQueue]
maxSize = 100MB

[queue=typingQueue]
maxSize = 10GB

[queue=tcpout_my_indexers]
maxSize = 100MB

[sslConfig]
sslPassword = $1$4h6Yj+pSDgCj
allowSslCompression=false

[lmpool:auto_generated_pool_download-trial]
description = auto_generated_pool_download-trial
quota = MAX
slaves = *
stack_id = download-trial

[lmpool:auto_generated_pool_forwarder]
description = auto_generated_pool_forwarder
quota = MAX
slaves = *
stack_id = forwarder

[lmpool:auto_generated_pool_free]
description = auto_generated_pool_free
quota = MAX
slaves = *
stack_id = free

[license]
master_uri = https://10.55.3.23:8089

[diskUsage]
minFreeSpace = 500
Looks ok to me

0 Karma

nickhills
Ultra Champion

Unless your post has been cropped, you have no cluster config there at all.
You should have a section like:

[clustering]
master_uri = xxxxx:xxx
pass4SymmKey = xxxx
...
...

How did you configure the peers, through the UI, or via the cli with something like
splunk edit cluster-config -mode slave -master_uri https://10.160.31.200:8089 -replication_port 9887 -secret your_key?
https://docs.splunk.com/Documentation/Splunk/7.0.1/Indexer/ConfigurepeerswithCLI

If my comment helps, please give it a thumbs up!

schose
Builder

search have a look at peer nodes

cat splunkd.log |grep -i error

or provide log

0 Karma

ashishlal82
Explorer

its a windows box

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to May Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars this month. This ...

They're back! Join the SplunkTrust and MVP at .conf24

With our highly anticipated annual conference, .conf, comes the fez-wearers you can trust! The SplunkTrust, as ...

Enterprise Security Content Update (ESCU) | New Releases

Last month, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...