I am facing a problem in setting up a cluster in splunk...
I have 1 master splunk instance with 2 replication factor and 2 search factor, i have 3 splunk instance as peer node and 1 splunk instance as search head.
Now i have configured a load balancing forwarder for peer machines..as soon as i get data in file configured in forwarder, i get data spread on peer node(some data on some peer node while other on other peer node).
On the search head node and master node i get data from all peer node.But on the peers node i can see data specific to only that peer node,is that the way it should actually be..or their is some problem ?
Also the problem is that as my one of the peers get down,its data is erased from the search head node and master,but when the peer node is up the data comes back on the seach head and master.My problem is that in case when the peer is down.i am not getting data of the downed peer node..Please help me is this case.
Thanks in advance..
Regarding your question, "But on the peers node i can see data specific to only that peer node,is that the way it should actually be..or their is some problem?" I assume you're running a search directly on the peer node. If that's the case, you will only see the searchable data on that specific node. The peer cannot access data on another node. See:
Regarding your second question on the downed peer node: when a peer node goes down, you can temporarily lose access to some data while bucket-fixing is occurring. However, if you have a search factor of 2 and only one peer goes down, then the full data set should be available again very quickly. The master just needs to reassign the primaries from the downed node to searchable copies on other nodes.
Thanks for ur answer satisfied with the answer of first question. But I am still not getting the solution for second question as I am not getting the data of downed peer on the search head or master even after waiting for some time..
Can you ply tell if their any configuration to be done on master,search head or peer node so that it could work fine.