On a healthy index, these two queries return the same value, or at least very similar, since the value is changing as data is indexed:
|metadata type=sourcetypes | stats sum(totalCount)
|dbinspect | stats sum(eventCount)
metadata seems to use the files at
*/db/*.data
dbinspect seems to use the files one level down at
*/db/*/*.data
I believe the rebuild command can be used to rebuild the .data on a bucket by bucket basis. Is there a similar command for rebuilding the .data files at the index level, the .data files just inside db?
This is NOT supported, but should work...
An answer I was given off-board was to move the *.data files at the index level aside and restart. This seems to rebuild those files from the *.data files in the buckets themselves.
It would be nice to have a simple way to rebuild all counts, in all buckets and at the index level.
The "root directory of the index" is e.g. $SPLUNK_DB/defaultdb/db/ ($SPLUNK_DB/defaultdb/ will NOT work). With Splunk 7, meta.dirty is deleted from db/ upon restart but the index is not rebuilt.
I found the following method on https://answers.splunk.com/answers/72562/how-to-reindex-data-from-a-forwarder.html (dating back to 2013):
1) # splunk stop
2) # splunk clean eventdata -index main
This sort of worked, except older data did not get re-indexed. My horizon shrunk from several days to about 5 hours. It ended up easier to remove the data sources (which were directories under surveillance anyway) and add them back in.