how to find out why an indexer is using more license than other indexers? Because i have 5 indexers, out of which 2 indexers were using 12% whereas other 3 indexers were using 11% and license pool quota exceeded. I am trying to figure it out what's going on with these indexers and why they are using more licensing?
You could have DBConnect running on an indexer and indexing locally. If this is the case, you should move DBConnect to a Heavy Forwarder that send to the entire Indexer Tier.
Hi kteng2024, as somesoni2 alluded to in his comment, my first guess is that there are probably some hosts that are only sending to certain indexers. You can get a count how many incoming connections there are by indexer by doing something like:
index=_internal host=YOUR_INDEXER_NAMES group=tcpin_connections | stats dc(sourceHost) as "Connecting Clients" by host
With proper load balancing setup, over an adequately long time period ( a day?), you should see about the same number of distinct connections for each indexer. If one looks particularly off, you can get list to diff against the other indexers by doing:
index=_internal host=YOUR_INDEXER_NAMES group=tcpin_connections | stats values(sourceHost) as "Connecting Client LIst" by host
Alternatively you could have other inputs besides splunk 2 splunk (HTTP Event Collector maybe?)
Please let me know if this answers your question! 😄
i see that indexing rate is equal on all the indexers . But still trying to figure out what might be the reason .
I would check if all forwarders are using proper load balancing across all indexers. Few good tips are discussed in following topic
https://answers.splunk.com/answers/62908/universal-forwarder-not-load-balancing-to-indexers.html
Use this link for more information on load balancing
https://docs.splunk.com/Documentation/Splunk/6.5.2/Forwarding/Setuploadbalancingd