Unable to run any search query : WARN: Search filters specified using splunk_server/splunk_server_group do not match any search peer.

[splunker12er]

WARN: Search filters specified using splunk_server/splunk_server_group do not match any search peer.

Possibilities :
relax the primary search criteria -> (index=* doesnt work)
widen the time range of the search ->(time range chosen in 'all time')
check that the default search indexes for your account include the desired indexes -> (admin role -> using default settings)

what could be the cause ?

Splunk version: Splunk 6.0.4 (build 207768)
Role : License master servers
Slaves version: Splunk 6.2.1 (build 245427)

[openpath_llc]

Encountered this same bug on Splunk 8.0.2.1. The steps from @ii_splunk worked well for me also.

[marcoscala]

Same bug on 8.0.8. The workaround proposed worked!!!

[terminaloutcome]

Same, on 8.0.1.

[ii_splunk]

I think this is a bug that Splunk needs to fix.... here is the work around in case anyone gets this:

On your search head do the following:

Settings->Distributed Management Console
(NOTE: Indexers will have N/A shown)
Setup->Apply Changes->Refresh
(NOTE: No changes were actually made)

Verify fix by clicking "Overview" in Distributed Management Console; Indexers will now show correct indexing rate.

Search as normal; workaround complete.

[lycollicott]

ii_splunk,
Why and how does that work? It worked for me, but I don't understand it at all.

Settings->Distributed Management
Console (NOTE: Indexers will have N/A
shown) Setup->Apply Changes->Refresh
(NOTE: No changes were actually made)

Verify fix by clicking "Overview" in
Distributed Management Console;
Indexers will now show correct
indexing rate.

[triest]

Of particular note is that this affected all searches.

As far as I know no changes where made to our DMC setup; we noticed that all searches quit working on our cluster master with the above mentioned error message.

[yannK]

Here is the known bug SPL-99116

After enabling the Distributed Management Console DMC, in "distributed mode", in an indexing cluster, the search-head may not be able to search all the peers. The error will mention splunk_server_group : "Search filters specified using splunk_server/splunk_server_group do not match any search peer". The workarounds are to go to the DMC setup page and hit "apply". To avoid the issue switch the DMC to "single instance" mode.

http://docs.splunk.com/Documentation/Splunk/6.2.2/ReleaseNotes/KnownIssues#Distributed_search_and_se...

[MuS]

Hi ii_splunk & kylekoza,

please file a bug report with Splunk Support if this is re-producable http://docs.splunk.com/Documentation/Splunk/6.2.0/Troubleshooting/HowtofileagreatSupportcase
But to be honest - I believe you had some trouble - this question is not related to Distributed management console. DMC is only available since Splunk 6.2 http://docs.splunk.com/Documentation/Splunk/6.2.0/ReleaseNotes/MeetSplunk#Distributed_management_con... and @splunker12er is using Splunk 6.0.4

cheers, MuS

[ii_splunk]

I can't reproduce at will but when the cluster get's in this "odd" state; I happened onto this work around. Has reoccured a few times on our cluster.

[kylekoza]

I had the same issue and this fixed it. Thanks!

[ridwanahmed]

thank you! I had the same ridiculous issue haha

[Lucas_K]

try putting splunk_server=* into your base search.

I just encountered this on a hunk install.

[MuS]

Hi splunker12er,

It is I again 😉

Does your License master, where you run this search, have any search peers configured? Check in the UI

http[s]://YourSplunkHostName:YourSplunkPort/en-GB/manager/search/search/distributed/peers

or by using this REST command on the license master:

| REST /services/search/distributed/peers

cheers, MuS