Deployment Architecture

The percentage of small buckets exceeded

michel_wolf
Path Finder

Hello guys,

I checked the splunk answers but I can´t find a solution for my problem. I have an indexer cluster with 2 idx and 2 sites and for my _internal index I get many small buckets. In the answers I found some notes about connection issues, but in this environment I don´t have connection problems.

All splunk instances are installed in 7.3.3

I get the following error:
alt text

I checked with the |dbinspect my _internal index and didn´t find any issues here:
|dbinspect index=_internal
|fields - splunk_server
|table startEpoch endEpoch *

alt text

Do you have any hints for me why new buckets are generated instead of using the existing one?
I don´t change many things on the default configuration for the _internal index:

/opt/splunk/bin/splunk cmd btool indexes --debug list _internal

/opt/splunk/etc/slave-apps/customer_all_indexes/local/indexes.conf [_internal]
/opt/splunk/etc/system/default/indexes.conf archiver.enableDataArchive = false
/opt/splunk/etc/system/default/indexes.conf archiver.maxDataArchiveRetentionPeriod = 0
/opt/splunk/etc/system/default/indexes.conf assureUTF8 = false
/opt/splunk/etc/system/default/indexes.conf bucketRebuildMemoryHint = auto
/opt/splunk/etc/slave-apps/customer_all_indexes/local/indexes.conf coldPath = volume:main/_internaldb/colddb
/opt/splunk/etc/system/default/indexes.conf coldPath.maxDataSizeMB = 0
/opt/splunk/etc/system/default/indexes.conf coldToFrozenDir =
/opt/splunk/etc/system/default/indexes.conf coldToFrozenScript =
/opt/splunk/etc/system/default/indexes.conf compressRawdata = true
/opt/splunk/etc/system/default/indexes.conf datatype = event
/opt/splunk/etc/system/default/indexes.conf defaultDatabase = main
/opt/splunk/etc/system/default/indexes.conf enableDataIntegrityControl = false
/opt/splunk/etc/system/default/indexes.conf enableOnlineBucketRepair = true
/opt/splunk/etc/system/default/indexes.conf enableRealtimeSearch = true
/opt/splunk/etc/system/default/indexes.conf enableTsidxReduction = false
/opt/splunk/etc/system/default/indexes.conf fileSystemExecutorWorkers = 5
/opt/splunk/etc/system/default/indexes.conf frozenTimePeriodInSecs = 2592000
/opt/splunk/etc/slave-apps/customer_all_indexes/local/indexes.conf homePath = volume:main/_internaldb/db
/opt/splunk/etc/system/default/indexes.conf homePath.maxDataSizeMB = 0
/opt/splunk/etc/system/default/indexes.conf hotBucketTimeRefreshInterval = 10
/opt/splunk/etc/system/default/indexes.conf indexThreads = auto
/opt/splunk/etc/system/default/indexes.conf journalCompression = gzip
/opt/splunk/etc/system/default/indexes.conf maxBloomBackfillBucketAge = 30d
/opt/splunk/etc/system/default/indexes.conf maxBucketSizeCacheEntries = 0
/opt/splunk/etc/system/default/indexes.conf maxConcurrentOptimizes = 6
/opt/splunk/etc/system/default/indexes.conf maxDataSize = 1000
/opt/splunk/etc/system/default/indexes.conf maxGlobalDataSizeMB = 0
/opt/splunk/etc/system/default/indexes.conf maxGlobalRawDataSizeMB = 0
/opt/splunk/etc/system/default/indexes.conf maxHotBuckets = 3
/opt/splunk/etc/system/default/indexes.conf maxHotIdleSecs = 0
/opt/splunk/etc/system/default/indexes.conf maxHotSpanSecs = 432000
/opt/splunk/etc/system/default/indexes.conf maxMemMB = 5
/opt/splunk/etc/system/default/indexes.conf maxMetaEntries = 1000000
/opt/splunk/etc/system/default/indexes.conf maxRunningProcessGroups = 8
/opt/splunk/etc/system/default/indexes.conf maxRunningProcessGroupsLowPriority = 1
/opt/splunk/etc/system/default/indexes.conf maxTimeUnreplicatedNoAcks = 300
/opt/splunk/etc/system/default/indexes.conf maxTimeUnreplicatedWithAcks = 60
/opt/splunk/etc/system/default/indexes.conf maxTotalDataSizeMB = 500000
/opt/splunk/etc/system/default/indexes.conf maxWarmDBCount = 300
/opt/splunk/etc/system/default/indexes.conf memPoolMB = auto
/opt/splunk/etc/system/default/indexes.conf minHotIdleSecsBeforeForceRoll = auto
/opt/splunk/etc/system/default/indexes.conf minRawFileSyncSecs = disable
/opt/splunk/etc/system/default/indexes.conf minStreamGroupQueueSize = 2000
/opt/splunk/etc/system/default/indexes.conf partialServiceMetaPeriod = 0
/opt/splunk/etc/system/default/indexes.conf processTrackerServiceInterval = 1
/opt/splunk/etc/system/default/indexes.conf quarantineFutureSecs = 2592000
/opt/splunk/etc/system/default/indexes.conf quarantinePastSecs = 77760000
/opt/splunk/etc/system/default/indexes.conf rawChunkSizeBytes = 131072
/opt/splunk/etc/slave-apps/_cluster/default/indexes.conf repFactor = auto
/opt/splunk/etc/system/default/indexes.conf rotatePeriodInSecs = 60
/opt/splunk/etc/system/default/indexes.conf rtRouterQueueSize = 10000
/opt/splunk/etc/system/default/indexes.conf rtRouterThreads = 0
/opt/splunk/etc/system/default/indexes.conf selfStorageThreads = 2
/opt/splunk/etc/system/default/indexes.conf serviceInactiveIndexesPeriod = 60
/opt/splunk/etc/system/default/indexes.conf serviceMetaPeriod = 25
/opt/splunk/etc/system/default/indexes.conf serviceOnlyAsNeeded = true
/opt/splunk/etc/system/default/indexes.conf serviceSubtaskTimingPeriod = 30
/opt/splunk/etc/system/default/indexes.conf splitByIndexKeys =
/opt/splunk/etc/system/default/indexes.conf streamingTargetTsidxSyncPeriodMsec = 5000
/opt/splunk/etc/system/default/indexes.conf suppressBannerList =
/opt/splunk/etc/system/default/indexes.conf suspendHotRollByDeleteQuery = false
/opt/splunk/etc/system/default/indexes.conf sync = 0
/opt/splunk/etc/system/default/indexes.conf syncMeta = true
/opt/splunk/etc/slave-apps/customer_all_indexes/local/indexes.conf thawedPath = $SPLUNK_DB/_internaldb/thaweddb
/opt/splunk/etc/system/default/indexes.conf throttleCheckPeriod = 15
/opt/splunk/etc/system/default/indexes.conf timePeriodInSecBeforeTsidxReduction = 604800
/opt/splunk/etc/system/default/indexes.conf tsidxReductionCheckPeriodInSec = 600
/opt/splunk/etc/system/default/indexes.conf tsidxWritingLevel = 1
/opt/splunk/etc/slave-apps/customer_all_indexes/local/indexes.conf tstatsHomePath = volume:main/_internaldb/datamodel_summary
/opt/splunk/etc/system/default/indexes.conf warmToColdScript =

Thanks

Michel

0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...