I want to carry out performance monitoring of Splunk. I came across this benchmark while browsing https://docs.splunk.com/Documentation/Splunk/6.1.14/Installation/Referencehardware. What was the configuration in config files like limits.conf and indexes.conf while carrying out these measurements? Were some parameters like memPoolMB or max_mem_usage_mb tuned to achieve better performance or were the set to their default configuration only?
The Reference hardware and sizing guides are based on not making any changes to limits.conf.
Adjustments to limits tends to be tuning around large event types, lookups or other environmental peculiarities you have in your environment. the default limits.conf should be optimized for a 'standard' Splunk deployment.
The same is true for indexes.conf, with some subtle nuances.
The reference spec includes storage with a defined rate of IOPS.
You should configure your hot/warm data to exist on your fastest storage device, as you want to be able to read/write as quickly as possible.
Your use case for data life-cycle management dictates after how old(big) your warm data gets before it gets moved to cold.
If you have two tiers of storage (ssd & spinning disk) there are advantages to separating the index paths, putting cold on the slower disk, but the reference indexing capabilities are primarily concerned with the indexers ability (and the IOPS) taken to write hot/warm.