Getting Data In

Splunk indexers IOPS tade off

egsub
Explorer

Hi,

We're considering to raise the number of indexers servers (peers) with reducing the IOPS per server FS due to IOPS limitations in our hardware.

Does this calculation fit? For instance, does 10 servers with 600 IOPS S-ATA based are equal to 5 servers with 1200 IOPS SAS based in case of 500GB/day indexed data?

Thanks,
EG

Tags (3)
0 Karma

esix_splunk
Splunk Employee
Splunk Employee

It should also be noted that 900iops is the minimum recommend spec for Splunk 6.2+. If you arent able to meet these requirements and decide to deploy lower spec hardware, you need to be careful and carefully monitor the Splunk Queues. Most likely you're going to see issues and need to modify configurations on your outputs to the indexers etc.

ekost
Splunk Employee
Splunk Employee

There's no magic answer available, as the type of search and the data volume per day will have a large impact on relative performance. If your SATA-based storage delivers the promised 600 IOPS (test with bonnie++,) and the type of searches you are running are balanced between CPU and I/O bound, and the data volume per-day per-indexer averages ~100GB/Day, and the data coming from the forwarders is evenly distributed across all indexers, and assuming an identical CPU core count, then having 2x indexers with less IOPS should equal your other hardware. There is no substitute for testing with actual data and a search and indexing load. Please note that a lot of admins discover that poor search result speed causes user aggravation, and insufficient IOPS can be a factor.

jmheaton
Path Finder

You should be more than fine with going with 5 servers at 1200 SAS.
A good rule of thumb is to go 1 server, 8 cores, 8 GB ram, and 1000 IOPS per 100GB daily data.
Do you actually index all 500GB/day

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...