Give this a try.
index=_internal sourcetype=splunkd component=TcpOutputProc (host=host1 OR host=host2....) earliest=-30m | eval Indexer=mvindex(split(idx,":"),0) | stats dc(Indexer) as Idx_Count values(Indexer) as Indexers by host
This will get you the indexers a forwarder (specified as host=...) is connecting to with their names. Now you add appropriate where clause compare it against. Another variations is below, which add column to all available indexers (which are added as distributed search peers).
above search | appendcols[| rest /services/search/distributed/peers | table title | eval Indexer=mvindex(split(title,":"),0) | stats values(Indexer) as AllIndexers dc(Indexer) as TotalIndexers ]
@ddrillic,
i am positive there are other ways to do it but here is a quick and dirty solution:
lets assume you have 3 indexers
you can search:
index = * | timechart span=5m dc(splunk_server) as unique_indexers by host
every host that has shows less then 3 in the chart is a suspect.
remember however that this test is not 100% accurate as maybe sometimes there is no new data for a while and therefore the forwarders will not ave much data to send.
i think that the best is to verify that you have the correct outputs.conf on your forwarders and verify the
forceTimebasedAutoLB = true
then you can enforce:
autoLBFrequency = <seconds>
or
autoLBVolume = <bytes>
read here more:
https://docs.splunk.com/Documentation/Splunk/6.6.2/Admin/Outputsconf
hope it helps
Gorgeous!!!!
Give this a try.
index=_internal sourcetype=splunkd component=TcpOutputProc (host=host1 OR host=host2....) earliest=-30m | eval Indexer=mvindex(split(idx,":"),0) | stats dc(Indexer) as Idx_Count values(Indexer) as Indexers by host
This will get you the indexers a forwarder (specified as host=...) is connecting to with their names. Now you add appropriate where clause compare it against. Another variations is below, which add column to all available indexers (which are added as distributed search peers).
above search | appendcols[| rest /services/search/distributed/peers | table title | eval Indexer=mvindex(split(title,":"),0) | stats values(Indexer) as AllIndexers dc(Indexer) as TotalIndexers ]
Just realized, the first search gives IP addresses, so if you're just comparing count, you can use above query as is. If you're comparing Indexer names as well, you need to do a dnslookukp before the stats.
Amazing !!!
We see the AllIndexers
column for the first row only out of two...
Yes. The rest is returning just single row. Just add |filldown
at the end so it'll get copied to all the rows.
do you mean if the forwarders sends data to all relevant indexers?
Exactly that....