Deployment Architecture

What would trigger Indexer Cluster Master restart?

tkw03
Communicator

So my indexer cluster master seems to be reloading all of the buckets. I didn't initiate a restart of the master or a rolling restart of the cluster. Is there anything else that would trigger either automatically?

Thanks for the input

0 Karma
1 Solution

mdillon_splunk
Splunk Employee
Splunk Employee

Is it possible you had a network interruption from your Cluster Master ?
Has this situation only happened the once ?
Check for disk I/O errors, or NIC errors on your Cluster Master around the time the peers were being added back.

Note: Cluster heartbeats have improved with more recent Splunk versions:

https://conf.splunk.com/files/2017/slides/scaling-indexer-clustering-5-million-unique-buckets-and-be...

View solution in original post

mdillon_splunk
Splunk Employee
Splunk Employee

Is it possible you had a network interruption from your Cluster Master ?
Has this situation only happened the once ?
Check for disk I/O errors, or NIC errors on your Cluster Master around the time the peers were being added back.

Note: Cluster heartbeats have improved with more recent Splunk versions:

https://conf.splunk.com/files/2017/slides/scaling-indexer-clustering-5-million-unique-buckets-and-be...

dkeck
Influencer

HI,

what do you mean by "reloading buckets". What is the master doing that it shouldn´t?

0 Karma

tkw03
Communicator

What I mean is it looks like the master got restarted as it went into making peers check back in and re-meeting rep factor and search factor

0 Karma

dkeck
Influencer

Did you confirm that it wasn´t restarted in the first place?

index=_internal source=splunkd.log "(build"

0 Karma

tkw03
Communicator

No restart, no results

So why would it resync with all peers?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...