I'm running some tests with searchhead pooling and scheduled searches. I'm running a saved search every minute and the results are being sent from one search head only. They're only coming from the second search head when the first search head is down (restarted, etc.). My question is: how does Splunk determine which searchhead to run a scheduled search from? Is this configurable (round-robin, least connections, pegged to specific host, etc.)?
Are the clocks on your two search heads in sync?
The search heads compete to run scheduled searches, thus there is no deterministic way of determining which search head will run a scheduled search (if at all). The workload should start to spread evenly across the search heads as the number of scheduled searches increases. Is there any reason that you're looking to configure on which search head a scheduled search runs?