How can I query Splunk to tell me how much space it thinks is being used in each volume? My volumes have nothing but Splunk data in them, and are entire partitions. I want to know how close I am to my set maxVolumeDataSizeMB value.
The only time that Splunk currently reports the size is when it rolls buckets due to enforcement of a volume max size policy. In such case, running the search index=_internal component=VolumeManager
would show you the current size and the max. To otherwise compute the volume size, the best thing to do is to run du -sh /volume/dir
(on Linux, or whatever the appropriate command is on your system to get the aggregate size). This should be pretty close to what Splunk considers to be the size.
You are correct. In the current version of the API (5.0.2), only total index usage, not per volume, is reported. Furthermore there is no interface that I am aware of that reports on volume usage (whether actual disk volumes or Splunk indexes.conf-defined logical volumes)
I'd like to hear more about this REST API
Based on this doc:
http://docs.splunk.com/Documentation/Splunk/latest/RESTAPI/RESTindex
It looks like it would only give you the total DB size, not the hot/warm separate from the cold. That would not help in my instance. Am I interpreting that correctly?
The du -sm solution does not work for me. The result comes in above my maxVolumeDataSizeMB. I have checked for indexes on that path that aren't configured to use the volume. (I had and fixed that problem before, as you can see here: http://splunk-base.splunk.com/answers/47963/mismatch-between-df-and-splunk-size-of-volume)
There does exist a Splunk REST endpoint that can provide the current size for each index as well. In that case, one could use that, in combination with the config settings that say which are configured on which volume (both pieces of data are at the REST API endpoint at /services/data/indexes
), to get a more accurate picture of what Splunk considers to be full "volumes". that is I believe the original motivation for the question, that what Splunk considers a "volume" and its "contents" differs from the actual filesystem volume and the files that are there.
good question.