splunk search for index's maximum configured size

[w531t4]

Hi all,

I found an answer here on the Splunk forums that shows a good search to list the current size of indexes as they sit on disk.. I would now like to associate these numbers with the MB size restrictions i have configured in indexes.conf per index..

Does anyone know of a good search that would produce theses values?

[w531t4 · Accepted solution]

"|btool indexes" definitely was the way to go. This is really what i was looking for.

| btool indexes 
| rex mode=sed "s/\r?\n/--BREAKER--/g" 
| rex field=_raw "(?<firstline>.+?)--BREAKER--(?<otherlines>.*)$" 
| eval otherlines=split(otherlines, "--BREAKER--")
| rex field=firstline ".*?\s+\[(?<indexname>.+)\]$"
| rex field=otherlines "(?<a>\S+)\s+(?<b>[^=]+)=(?<c>.*)" max_match=1 
| eval fields=mvzip(a,mvzip(b,c))
| mvexpand fields 
| rex field=fields "^(?<filename>[^,]+),(?<k>[^,]+),(?<v>.*)" 
| table filename,k,v,sos_server,indexname
| where k like "%maxTotalDataSizeMB%"

[aholzer]

Very cool. Didn't realize the btool function was available at search time. Thought it was just a CLI thing. Glad you found the answer 🙂

[w531t4]

gotcha -- looks like |btool indexes is the way to go!

[aholzer]

Well... you could write a script that monitors your indexes.conf, and aggregates the max_sizes for you and then gets indexed in say main. Then you could simply search against that data rather than maintaining a lookup.

Or you could write a script that writes your lookup csv and runs automatically on a schedule, therefore removing the need for manual intervention.

There may be a way of doing it out of the box, it just escapes me. There are people far more knowledgable than me on here though, and one of them might take a look at your question and chime in with a brilliant answer 🙂

[w531t4]

OK - I suppose i was originally hoping that I could pull the configuration values out rather than maintaining an the index sizes in both indexes.conf and a lookup csv. But, i guess it is what it is. Thanks for the help.

[aholzer]

I'd suggest you create a csv with two columns: index and max_size

Then use lookup GUI interface to create a lookup table and definition with this data (index_size.csv and index_size). You can then use the lookup command to get the max_size from the table and link it with your search. Like so:

| eventcount summarize=false report_size=true index=*
| eval MB = size_bytes / 1024 / 1024
| lookup index_size index OUTPUT max_size

This will then append the column "max_size" from your file to your results.

Hope this helps

[w531t4]

my fault.

| eventcount summarize=false report_size=true index=*
| eval MB = size_bytes / 1024 / 1024

[aholzer]

Going forward it would be helpful if you add a link to the other answer you found, or put the search you want to upgrade in your question.