Getting Data In

Need to set an upper size limit across multiple volumes

jducote
New Member

I need to set an upper size limit across multiple volumes, for indexing, regardless of hot/warm/cold status. It is my understanding that I may do this by setting the homePath.maxDataSizeMB switch in indexes.conf. Do I need to establish the three volumes individually, or will setting (in example) homePath.maxDataSizeMB=850000[MB] limit each registered volume as having an upper limit of 850GB? Or do I need to specify the limit for each volume, in the same file?

Some background:

3 Volumes of 1TB (986GB usable):
/dev/md127 985G 897G 38G 96% /opt/splunk
/dev/md125 985G 394G 541G 43% /opt/splunk-index
/dev/md126 985G 240G 695G 26% /opt/splunk-index2

As you can see, we are nearly full on the first volume. Pruning index size was successful, but mostly affected /opt/splunk-index. My desired state would be to have data more evenly distributed amongst the three volumes.

0 Karma

yannK
Splunk Employee
Splunk Employee

If you want to control the size per volume instead of per index, check this guide
http://docs.splunk.com/Documentation/Splunk/latest/Indexer/Configureindexstoragesize

The other way is more delicate, you can be very specific for each index using the number of hot and warm buckets to spread the buckets between homePath and coldPath.
http://wiki.splunk.com/Deploy:BucketRotationAndRetention

0 Karma

jducote
New Member

Thanks, it looks like using the homePath.maxDataSizeMB=X will be the most elegant option, per:
"Space taken by the whole index
The maximum size of all buckets for an index is maxTotalDataSizeMB. This is the size of hot+warm+cold. If you have hot+warm and cold on different volumes, try to maximize the use of the space." (http://wiki.splunk.com/Deploy:BucketRotationAndRetention)

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...