Monitoring Splunk

Multiple indexes + disk space usage

skippylou
Communicator

So if I follow the data space and retirement process correctly, it works in a circular manner with old data being deleted (as its frozen) as needed to make room for new data. With the default setting of 500GB max index size.

My question would be say you had two indexes, index1 with a max size setting of 400GB and index2 with a max size setting of 200GB, and you have 400GB of usable space. Let's say at some point you have 200GB used in index1 and you hit 200GB on index2. So index2 should be working in a circular fashion at this point rotating old data out. But what happens with index1? It is at only half its max, but the disk is full. Does it do circular based on this or does no old data get rotated out until an age-based policy applies? Should the total max index sizes never equal more than your total disk space available?

What is the best way to manage max index sizes on multiple indexes and also to ensure that any new data coming in regardless of the index makes it into its index?

Thanks,

Scott

Tags (2)
1 Solution

Genti
Splunk Employee
Splunk Employee

Scott,

Yes, total max index sizes should never equal more then your total disk space available.
In your described scenario, when diskspace is 400GB and Index1 has 200GB and index2 has also 200GB indexing will be stopped until you release at least 2GB(definitely more then 2 though) space.

Best way would be: Index A + B + C + D +... < TOTAL HD Space

View solution in original post

Genti
Splunk Employee
Splunk Employee

Scott,

Yes, total max index sizes should never equal more then your total disk space available.
In your described scenario, when diskspace is 400GB and Index1 has 200GB and index2 has also 200GB indexing will be stopped until you release at least 2GB(definitely more then 2 though) space.

Best way would be: Index A + B + C + D +... < TOTAL HD Space

skippylou
Communicator

Good to know, thanks Genti!

0 Karma
Get Updates on the Splunk Community!

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...

Introducing Splunk Enterprise 9.2

WATCH HERE! Watch this Tech Talk to learn about the latest features and enhancements shipped in the new Splunk ...

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...