- Splunk Answers
- :
- Splunk Administration
- :
- Getting Data In
- :
- What happens when I update maxTotalDataSizeMB in a...
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark Topic
- Subscribe to Topic
- Mute Topic
- Printer Friendly Page
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi guys,
We are running a multi site index cluster with 12 indexers (6 across 2 sites). Our goal is to limit the size of one of our indexes.
We currently have an index that's sitting at 150gb and I was planning on using the maxTotalDataSizeMB to limit it to around 75gb.
The remaining 75gb should go to frozen, is this correct? (We don't actual freeze data so I assume it just gets deleted)
If i apply this update to maxTotalDataSizeMB when the cluster is up and running, will it just delete the data will carrying on with regular tasks as normal? As it is our production environment, we can't take the cluster down for even a minute. (rolling restarts are okay as it'll need one when I apply the new cluster bundle).
Does anyone know if its okay to apply this config while the cluster is up and running?
Cheers
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Yes, this particular change may not even trigger a rolling restart when you apply the cluster bundle. The excess buckets will be frozen (deleted), starting with the oldest buckets. Just to clarify the behavior for other readers: maxTotalDataSizeMB applies to a specific index for each cluster member individually, not to the cluster as a whole. In this case with 12 indexers, setting a limit of 75 GB will result in total size of 900 GB across the cluster for the affected index.
This limit includes replicated buckets so cluster member failure or increasing replication factor could result in unexpected data loss. For this reason among others, I personally prefer to treat maxTotalDataSizeMB as a sort of emergency limit, using frozenTimePeriodInSecs as the primary tool to manage index size.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hey @robbie1194, same as the last post! If this solution worked, please accept the answer. 🙂
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Yes, this particular change may not even trigger a rolling restart when you apply the cluster bundle. The excess buckets will be frozen (deleted), starting with the oldest buckets. Just to clarify the behavior for other readers: maxTotalDataSizeMB applies to a specific index for each cluster member individually, not to the cluster as a whole. In this case with 12 indexers, setting a limit of 75 GB will result in total size of 900 GB across the cluster for the affected index.
This limit includes replicated buckets so cluster member failure or increasing replication factor could result in unexpected data loss. For this reason among others, I personally prefer to treat maxTotalDataSizeMB as a sort of emergency limit, using frozenTimePeriodInSecs as the primary tool to manage index size.
- Mark as New
- Bookmark Message
- Subscribe to Message
- Mute Message
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
If you use Distribute Configuration Bundle from the master, you should be just fine.
Threat Hunting Unlocked: How to Uplevel Your Threat Hunting With the PEAK Framework ...
Splunk APM: New Product Features + Community Office Hours Recap!
Index This | Forward, I’m heavy; backward, I’m not. What am I?
