Deployment Architecture

Distributed Deployment: Splunk data replication

MHibbin
Influencer

Hi,

I have a question with a distributed deployment...

If a deployment was set-up to have for example:

  • 2 x Indexers
  • n x Forwarders (set-up to autoLb between the indexers)
  • 1 x Search Head

The autoLB will forward data to the Splunk indexers in a cycle based on time.What happens to the visibility of data if one of the Indexers was to become inactive (e.g. a system failure, etc). I would imagine that Splunk would be able to view ~half of the data, is this assumption correct?

How would data replication between the Indexers take place? - If the there is a requirement for the data to remain 100% visible, what would be best to achieve this?

I'm sure I have come across guidelines on data replication between two indexers in past notes/discussions/Splunk documentation. But I am not able to find the justification I require.

Are there any thoughts on documentation or sources of information that would be useful?

Any thoughts welcome, thanks in advance.

Regards,

MHibbin

1 Solution

gkanapathy
Splunk Employee
Splunk Employee

You are correct that if one indexer is out, only half the data will be visible, though the search head will report that it is unable to reach all indexers.

In the current version, there is no native replication of data. You will have to do this either using the underlying storage to replicate, or by forwarding from indexer to a replica instance. Both have disadvantages relative to the other. In addition, there is no built-in mechanism for failover, so you would have to implement this yourself. These solutions are not entirely simple to implement correctly and robustly. An overview of this is here: http://docs.splunk.com/Documentation/Splunk/4.3.2/Installation/Highavailabilityreferencearchitecture

In future versions, you may expect some form of built-in replication, as well as a more automated built-in failover, that should be preferable to these other methods.

View solution in original post

gkanapathy
Splunk Employee
Splunk Employee

You are correct that if one indexer is out, only half the data will be visible, though the search head will report that it is unable to reach all indexers.

In the current version, there is no native replication of data. You will have to do this either using the underlying storage to replicate, or by forwarding from indexer to a replica instance. Both have disadvantages relative to the other. In addition, there is no built-in mechanism for failover, so you would have to implement this yourself. These solutions are not entirely simple to implement correctly and robustly. An overview of this is here: http://docs.splunk.com/Documentation/Splunk/4.3.2/Installation/Highavailabilityreferencearchitecture

In future versions, you may expect some form of built-in replication, as well as a more automated built-in failover, that should be preferable to these other methods.

Get Updates on the Splunk Community!

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...