Deployment Architecture

How is bucket deletion due to retention managed in a multisite indexer clustering environment?

rbal_splunk
Splunk Employee
Splunk Employee

I have a multisite cluster. When some of the buckets meet retention, will the bucket get deleted from all the Cluster Peers? How is the Bucket deletion managed due to Retention in a Multi_site Cluster?

1 Solution

rbal_splunk
Splunk Employee
Splunk Employee

Once a bucket is deleted from a peer, it notifies the Cluster Master, and as the Cluster Master marks it frozen, it then alerts to all other peers that it's 'frozen'. This secondary alert doesn't do anything to the bucket, but the remaining peers remember it's frozen for the purpose of a restart. When the bucket is re-added, the CM will still know it's frozen, otherwise the frozen flag may be lost.

Frozen buckets are deleted individually by their indexers. A frozen bucket on indexer-A deletes it off indexer-A, but will not delete it from indexer-B. This is by design. Retention policies kick in individually by indexer.

Once a bucket is frozen, it is marked as so and they no longer count towards a cluster's Searchability, Replication Factor, and Search Factor needs.
If the cluster is unsearchable or not meeting RF/SF requirements, it is not due to buckets marked as frozen.

View solution in original post

rbal_splunk
Splunk Employee
Splunk Employee

Once a bucket is deleted from a peer, it notifies the Cluster Master, and as the Cluster Master marks it frozen, it then alerts to all other peers that it's 'frozen'. This secondary alert doesn't do anything to the bucket, but the remaining peers remember it's frozen for the purpose of a restart. When the bucket is re-added, the CM will still know it's frozen, otherwise the frozen flag may be lost.

Frozen buckets are deleted individually by their indexers. A frozen bucket on indexer-A deletes it off indexer-A, but will not delete it from indexer-B. This is by design. Retention policies kick in individually by indexer.

Once a bucket is frozen, it is marked as so and they no longer count towards a cluster's Searchability, Replication Factor, and Search Factor needs.
If the cluster is unsearchable or not meeting RF/SF requirements, it is not due to buckets marked as frozen.

Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...