Deployment Architecture

What would trigger Indexer Cluster Master restart?

tkw03
Communicator

So my indexer cluster master seems to be reloading all of the buckets. I didn't initiate a restart of the master or a rolling restart of the cluster. Is there anything else that would trigger either automatically?

Thanks for the input

0 Karma
1 Solution

mdillon_splunk
Splunk Employee
Splunk Employee

Is it possible you had a network interruption from your Cluster Master ?
Has this situation only happened the once ?
Check for disk I/O errors, or NIC errors on your Cluster Master around the time the peers were being added back.

Note: Cluster heartbeats have improved with more recent Splunk versions:

https://conf.splunk.com/files/2017/slides/scaling-indexer-clustering-5-million-unique-buckets-and-be...

View solution in original post

mdillon_splunk
Splunk Employee
Splunk Employee

Is it possible you had a network interruption from your Cluster Master ?
Has this situation only happened the once ?
Check for disk I/O errors, or NIC errors on your Cluster Master around the time the peers were being added back.

Note: Cluster heartbeats have improved with more recent Splunk versions:

https://conf.splunk.com/files/2017/slides/scaling-indexer-clustering-5-million-unique-buckets-and-be...

dkeck
Influencer

HI,

what do you mean by "reloading buckets". What is the master doing that it shouldn´t?

0 Karma

tkw03
Communicator

What I mean is it looks like the master got restarted as it went into making peers check back in and re-meeting rep factor and search factor

0 Karma

dkeck
Influencer

Did you confirm that it wasn´t restarted in the first place?

index=_internal source=splunkd.log "(build"

0 Karma

tkw03
Communicator

No restart, no results

So why would it resync with all peers?

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...