Deployment Architecture

Index Cluster Scaling

FritzWittwer_ol
Contributor

Is there a limit for the number of nodes in an indexer cluster, above wich we will see a performance degradation? With our current growth rate we will probably come towards 128 indexers, each with 16 PB local storage, in a two site indexer cluster. There will also be a search head cluster with 20 to 30 search heads in tow different sites.

0 Karma

dxu_splunk
Splunk Employee
Splunk Employee

from a clustering perspective, as the number of buckets approaches 1 million + we start hitting some performance issues.

having said that, we've scaled to 1k+ indexers, and several million buckets+ recently with Splunk 6.6. there are some recommended timeouts we advocate raising for larger clusters (support should be able to help you out with that!)

FritzWittwer_ol
Contributor

Thanks for your reply. We are aware of the 1 million bucket limit, we already hit it and had to upgrade to 6.6 to overcome it

0 Karma
Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Wondering How to Build Resiliency in the Cloud?

IT leaders are choosing Splunk Cloud as an ideal cloud transformation platform to drive business resilience,  ...

Updated Data Management and AWS GDI Inventory in Splunk Observability

We’re making some changes to Data Management and Infrastructure Inventory for AWS. The Data Management page, ...