Getting Data In

Need to set an upper size limit across multiple volumes

jducote
New Member

I need to set an upper size limit across multiple volumes, for indexing, regardless of hot/warm/cold status. It is my understanding that I may do this by setting the homePath.maxDataSizeMB switch in indexes.conf. Do I need to establish the three volumes individually, or will setting (in example) homePath.maxDataSizeMB=850000[MB] limit each registered volume as having an upper limit of 850GB? Or do I need to specify the limit for each volume, in the same file?

Some background:

3 Volumes of 1TB (986GB usable):
/dev/md127 985G 897G 38G 96% /opt/splunk
/dev/md125 985G 394G 541G 43% /opt/splunk-index
/dev/md126 985G 240G 695G 26% /opt/splunk-index2

As you can see, we are nearly full on the first volume. Pruning index size was successful, but mostly affected /opt/splunk-index. My desired state would be to have data more evenly distributed amongst the three volumes.

0 Karma

yannK
Splunk Employee
Splunk Employee

If you want to control the size per volume instead of per index, check this guide
http://docs.splunk.com/Documentation/Splunk/latest/Indexer/Configureindexstoragesize

The other way is more delicate, you can be very specific for each index using the number of hot and warm buckets to spread the buckets between homePath and coldPath.
http://wiki.splunk.com/Deploy:BucketRotationAndRetention

0 Karma

jducote
New Member

Thanks, it looks like using the homePath.maxDataSizeMB=X will be the most elegant option, per:
"Space taken by the whole index
The maximum size of all buckets for an index is maxTotalDataSizeMB. This is the size of hot+warm+cold. If you have hot+warm and cold on different volumes, try to maximize the use of the space." (http://wiki.splunk.com/Deploy:BucketRotationAndRetention)

0 Karma
Get Updates on the Splunk Community!

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Wednesday, May 29, 2024  |  11AM PST / 2PM ESTRegister now and join us to learn more about how you can ...

Get Your Exclusive Splunk Certified Cybersecurity Defense Engineer Certification at ...

We’re excited to announce a new Splunk certification exam being released at .conf24! If you’re headed to Vegas ...

Share Your Ideas & Meet the Lantern team at .Conf! Plus All of This Month’s New ...

Splunk Lantern is Splunk’s customer success center that provides advice from Splunk experts on valuable data ...