Hi Splunk,
We will be using rsync to keep our 2 enterprise search heads in sync. I found a Splunk wiki page on this topic but it is for an older version (3.3) of Splunk. We are using Splunk 6.4. Is there a newer wiki or another documented resource on what to keep in sync between the 2 search heads?
Thank you
The following speaks about it - How configuration changes propagate in a search head cluster
This is not feasible, as there are only 2 servers. SHC requires 3.
Indeed, SHP (search head pooling) is no longer a viable option (as of 6.2). SHC (search head clustering) requires minimum 3 search heads. A better way of keeping things in sync between the search heads is to use a Deployment Server.
Deployment server can keep your Splunk Apps in Sync between the search heads, however, local
configurations will not be there. Same thing for users. That being said, what one search head does won't be transferred to another search head, and neither is aware of the other. Can you sync the dispatch
folder? Technically yes, however, the sids
won't exist on the other search head so it might get confused. So the best you can do is have 2 different search heads with the same users and apps. Any data or local changes would need to be rolled back into the DS to be pushed back down or rsynced. But don't rsync the search artifacts. It just won't work like you think it will.
To continue the discussion, join us on IRC (#splunk on EFNet), or Slack (www.splunk402.com/chat for access). Happy Splunking!
== EDIT ==
Changed the version on SHP from 6.3 to 6.2.
Thank you. When you say search artifacts do you mean the users history?
Example:
user/user1/search/history/searchhead.csv
I would join the IRC but work prohibits chat sessions.
Search artifacts are not just the user's history, but also remnants/results from previous searches. For example, if you have a scheduled search that runs and sends out a link to the results, that link is to search artifacts rather than rerunning the search.
Why are you not using the 2 supported options?
Search Head pooling:
http://docs.splunk.com/Documentation/Splunk/6.4.2/DistSearch/Configuresearchheadpooling
http://docs.splunk.com/Documentation/Splunk/6.4.2/DistSearch/Createasearchheadpool
Search Head clustering:
http://docs.splunk.com/Documentation/Splunk/6.4.2/DistSearch/AboutSHC
http://docs.splunk.com/Documentation/Splunk/6.4.2/DistSearch/SHCconfigurationoverview
http://docs.splunk.com/Documentation/Splunk/6.4.2/DistSearch/SHCarchitecture
Combined:
https://answers.splunk.com/answers/133653/search-head-pool-and-clustering.html
Migrating:
http://docs.splunk.com/Documentation/Splunk/6.4.2/DistSearch/Migratefromsearchheadpooling
Starting with 4.2, you can use search head pooling to share configurations and user data between search heads. So, the Search Head Pooling with LDAP authentication, you do not need to rsync the apps, users, and dispatch jobs among search heads, anymore.
Note: Search head pooling has been deprecated in Splunk Enterprise version 6.2.
Search Head Cluster takes care of this one.
For the love of Benji, don't do SHP.
I must admit, I have not done so myself, but surely it is better than rolling one's own rsync, yes?
Barely. SHP has a lot of performance problems in most cases, as it is CIFS/NFS based. Rsync provides different levels of pain. From what I have seen in the past, attempting to sync search heads via rsync produces weirdness (at best). Best option here, unfortunately, is to get another search head. Splunk recommends that servers in SHC be similar, which makes sense, as SHC doesn't really have a concept of "weighting". However, perhaps a small VM that sees no end-user searches.
OK, so just add the Deployment Server as the 3rd Search Head for Clustering but do not allow anybody to login to it. That should solve everything, right?
No, it still wouldn't be perfect. You still need a deployer, which would most likely be the DS. So another system would still be needed.
hello there,
we can't head SH clustering as we only have 2 search heads, and I read somewhere search head pooling was deprecated... I will check out your links.. thank you!
As Splunk says:
This feature has been deprecated as of Splunk Enterprise version 6.2. This means that although it continues to function, it might be removed in a future version.
So deprecated
does not mean unuseable
.