Deployment Architecture

When the hot bucket is created on clustered envioemnet. What steps are followed.

sat94541
Communicator

When the hot bucket is created on clustered envioemnet. What steps are followed.

0 Karma

rbal_splunk
Splunk Employee
Splunk Employee

When an indexer creates a hot bucket, it follows this procedure:

1) Indexer needs to create a new hot bucket
2) Indexer asks the CM whom to replicate the new hot bucket to
3) CM receives the request, checks the configured RF/SF, and selects indexers (randomly) as "targets" that the original indexer should stream to. For example, if RF=3, SF=2, it will respond with two targets, one of which will also be searchable (to satisfy SF=2).

All hot bucket replications are raw data ONLY. For the example above, the one target that is also told to be searchable will create its own tsidx files based on the rawdata that comes in.

rbal_splunk
Splunk Employee
Splunk Employee

Yes, we can survive failures, i.e. we won't stop indexing, but we cannot possibly meet replication policy for those source indexers that have a now failed peer in their target list. CM recovery and fixup needed asap, in that case
In other words: We really need CM HA.

0 Karma

sat94541
Communicator

Does splunk say that ..... if your CM is down....we can survive any additional failures

0 Karma

rbal_splunk
Splunk Employee
Splunk Employee

correct.in which case new buckets dont meet rf/sf

0 Karma

sat94541
Communicator

Until one of them fails as well, in which case.....?
The cluster cannot remain healthy until the CM comes back up...?

0 Karma

sat94541
Communicator

Until one of them fails as well, in which case.....?
The cluster cannot remain healthy until the CM comes back up...?

0 Karma

sat94541
Communicator

so with RF 3, it's going to be the same two indexers over and over

0 Karma

rbal_splunk
Splunk Employee
Splunk Employee

each indexer remembers the LAST list of targets the CM gave it. if the CM is down, it will continuously use the same targets for new hot buckets.
to be more correct, each indexer remembers the "last response of a new hot bucket request", and reuses that response

0 Karma

sat94541
Communicator

we've believe that if the CM is down, we don't have to really break our necks in bringing it back up, e.g. an hour or more will be OK; in this instance, does the indexer always use the same peers to replicate to, or does it have a list of them it can use?

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...