Deployment Architecture

Index Cluster Scaling

FritzWittwer_ol
Contributor

Is there a limit for the number of nodes in an indexer cluster, above wich we will see a performance degradation? With our current growth rate we will probably come towards 128 indexers, each with 16 PB local storage, in a two site indexer cluster. There will also be a search head cluster with 20 to 30 search heads in tow different sites.

0 Karma

dxu_splunk
Splunk Employee
Splunk Employee

from a clustering perspective, as the number of buckets approaches 1 million + we start hitting some performance issues.

having said that, we've scaled to 1k+ indexers, and several million buckets+ recently with Splunk 6.6. there are some recommended timeouts we advocate raising for larger clusters (support should be able to help you out with that!)

FritzWittwer_ol
Contributor

Thanks for your reply. We are aware of the 1 million bucket limit, we already hit it and had to upgrade to 6.6 to overcome it

0 Karma
Get Updates on the Splunk Community!

Everything Community at .conf24!

You may have seen mention of the .conf Community Zone 'round these parts and found yourself wondering what ...

Index This | I’m short for "configuration file.” What am I?

May 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with a Special ...

New Articles from Academic Learning Partners, Help Expand Lantern’s Use Case Library, ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...