Deployment Architecture

Search Head Clustering: How does the captain assign jobs to members with the "least search load" with no insight into CPU load?

a212830
Champion

I'm reading the docs for search-head-clustering, and trying to determine exactly how a job is assigned. The doc says that each job is assigned to the member currently with the least search load. But, it also says that captain has no insight into the CPU load. So, what is the captain looking at?

Steve_G_
Splunk Employee
Splunk Employee

The captain assumes that each member has the same resources (cpu, etc.). In assigning scheduled search jobs, it attempts to distribute jobs evenly among the members. So, it assigns each job to the member currently running the least number of jobs.

a212830
Champion

Thanks. I'd really like to see Splunk make the resource monitoring more intelligent. Most customers purchased large servers with lots of cpu's to handle the load (and to avoid SHP), and now we get this - which looks nice - but it seems that we are still limited to purchasing large servers. Let me add on and scale it horizontally - without needing to purchase extra large servers. Very frustrating.

0 Karma

Tanefo
Path Finder

hello,
if i understant very well your preoccupation, i think that:
- The capitain is a only member of cluster which is connected to Splunk (Deployer), it is a central distribution
the capitain schedules and manages searches

0 Karma

a212830
Champion

Thanks. I understand that it manages the scheduling, but how, exactly, if it's not looking at cpu load? How is it determining where jobs get run?

0 Karma

anz_leycurav
Explorer

As per this link: http://docs.splunk.com/Documentation/Splunk/6.2.3/DistSearch/PropagateSHCconfigurationchanges#Push_t...

"The -target parameter specifies the URI and management port for any member of the cluster, for example, https://10.0.1.14:8089. You specify only one cluster member but the deployer pushes to all members. This parameter is required."

So the captain isn't not even doing that.

i have established the required 3 SH members but cannot see a distribution of searches - each member appears to be doing all the searches. Maybe i'm just not looking in the right place - is there a way to monitor this closer?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...