Splunk Search

How to calculate the index size from all indexers

adityapavan18
Contributor

Is there a way to calculate total size of an index from all indexers?
I can see index size from each individual indexer but i have around 30 indexers in the setup i am working on..so ideally wouldn't work if go and check index size in each indexer individually

Tags (2)
1 Solution

alacercogitatus
SplunkTrust
SplunkTrust

Building on MuS' answer, you can do each indexer from a single search head. This will require SoS to be installed ( http://apps.splunk.com/app/748/ ).

| inputlookup splunk_servers_cache | search server_role="search-peer" | map maxsearches=100 search="| rest /services/data/indexes splunk_server=$sos_server$ " | stats sum(currentDBSizeMB) by title splunk_server

Each of these calls should be really quick, so the number of search-peers will increase execution time, but not as badly as if you were doing a data search.

You can also adjust the stats do whatever you need, | stats sum(currentDBSizeMB) by title.

View solution in original post

alacercogitatus
SplunkTrust
SplunkTrust

Building on MuS' answer, you can do each indexer from a single search head. This will require SoS to be installed ( http://apps.splunk.com/app/748/ ).

| inputlookup splunk_servers_cache | search server_role="search-peer" | map maxsearches=100 search="| rest /services/data/indexes splunk_server=$sos_server$ " | stats sum(currentDBSizeMB) by title splunk_server

Each of these calls should be really quick, so the number of search-peers will increase execution time, but not as badly as if you were doing a data search.

You can also adjust the stats do whatever you need, | stats sum(currentDBSizeMB) by title.

MuS
SplunkTrust
SplunkTrust

Just came along this old post, happy to give some karma to an old friend 

😊

0 Karma

dtrelford
Path Finder

SoS is discontinued and no longer supported. Is there an alternative query that can be run now?

My apologies if I should have posted this in a new question.

0 Karma

gjanders
SplunkTrust
SplunkTrust
| rest /services/data/indexes |  stats sum(currentDBSizeMB) by title splunk_server

The above might do it, you can use the REST API or use the monitoring console. The monitoring console has a variety of queries that do this which you can borrow by clicking on the magnifying glass (it replaces SoS)

alacercogitatus
SplunkTrust
SplunkTrust

You can even summarize this (with |collect index=summary) and then do data capacity planning!

0 Karma

adityapavan18
Contributor

Thanx a lot...I do have S.o.S installed..and running this query provided me the data i require

0 Karma

MHibbin
Influencer

The eventcount command may be what you need:

http://docs.splunk.com/Documentation/Splunk/6.0/SearchReference/Eventcount

Haven't tried it against multiple indexers however.

This command requires a "|" before the command to run.

Hope this helps.

MHibbin
Influencer

@adityapavan18, again I'm not sure, as I only one indexer to play around with at the moment, but perhaps the following, seems odd to me that it has the server field if it is localized to one server...

| eventcount summarize=false index=* report_size=true | eval MB=(size_bytes/1024)/1024 | stats sum(MB) by index, server

aokur_splunk
Splunk Employee
Splunk Employee

worked for me - thanks MHibbin

0 Karma

mendesjo
Path Finder

Nope doesn't work..

0 Karma

adityapavan18
Contributor

this helps only for a particular indexer.Not sure how we use for multiple indexers

0 Karma

MuS
SplunkTrust
SplunkTrust

Hi adityapavan18

one way would be to use the REST endpoint /services/data/indexes for that, but this must be done against each indexer. Personally I would setup a summarized saved search on each indexer which runs the following search:

     | rest /services/data/indexes | stats values(currentDBSizeMB) by title

This way you will be able to get the index size for each indexer with one single search afterwards.

hope this helps ...

cheers, MuS

awurster
Contributor

this is way better, as anything that requires the 3rd party app like SOS is not a clean one IMHO. i think more and more of these weird little tasks must be phased out and worked into core splunk (via REST API for instance or DMC moving forward.

0 Karma

adityapavan18
Contributor

Thanx MuS.

0 Karma

mendesjo
Path Finder

| rest /services/data/indexes | stats values(currentDBSizeMB) by title

That only provides data of index per indexer, it's not added up.. How would you sum it?

0 Karma

MuS
SplunkTrust
SplunkTrust

Quick and dirty solution is to run this:

 | rest /services/data/indexes | where totalEventCount!=0 | stats values(currentDBSizeMB) AS size by title | streamstats sum(size) AS total
0 Karma

sowings
Splunk Employee
Splunk Employee

Note that currentDBsizeMB is 1 for an index with no events. Might want to filter out those where totalEventCount=0

0 Karma

MuS
SplunkTrust
SplunkTrust

Good point, I changed the search - thanks 😉

0 Karma

Rocky31
Path Finder

what you changed? paste it here plz

0 Karma

Muryoutaisuu
Communicator

why not just use | stats sum(currentDBSizeMB) by title

instead of values?

0 Karma
Get Updates on the Splunk Community!

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...