Since Enterprise Security can only have 1 instance running, how do you handle high availability if that search head stops working?

[Splunker]

Hi all,

Have a 2 site distributed-architecture of Splunk, with 1 Search-Head in either site (and indexers and heavy-forwarders but right now am focusing on the search-heads). A fully virtualized solution (VMWare).

Since Splunk Enterprise-Security can only have 1 instance running on any particular deployment, how do folks handle high availability if that search head stops working?

There is a WAN between sites, so i've been told rsync is the best way (rather than SH pooling) which is fine, but i'd love to hear other peoples experiences with this type of setup.

Failing over, failing back, and any issues in between to be wary of. Also do you just rsync all of /opt/splunk/etc/* across, and leave Splunk not running on the warm standby instance until it's needed?

Would appreciate any advice..

Thanks.

[matthieu_araman]

Hello,

you could use the search head clustering functionality from Splunk 6.2
you'll need at least Splunk ES 3.2.1 + latest Splunk + meet requirements for SHC (for example, at least 3 servers, 4 (2 on each site) in some failover scenarios)
This would also divide the load between the servers.
Depending on your context, you should discuss the adapted requirements for SHC + ES with Splunk.

[PrinceOfEval]

I'm curious if you've found an answer to this. I've had discussions about it but never implemented anything. As I understand, a major issue with having non-pooled ES search heads is that the notable index is prone to confusion. Also, they would both need to build their own data model summaries, which may cause performance issues with indexers.