What is the best practice to perform a query using distributed search?

[danje57]

Hello Splunkers,

I've a issue with my distributed searches.

I've one search head and 2 indexers. Both indexers are configured as peers and can be reached by my Search Header.

However, I'm facing an issue where I have a query on the Search Head which returns nothing.

Then I try the same query on my Indexer and I've the expected result.

index=corp_mss_fileserver source="*Acl*Fileserver*.csv" | eval Path=replace("F:\Fileserver\5_IT_folder\000000","\\\\", "\\\\") | where FullName=Path | rex field=Account "CORPORATE\\\(?<Group>.*)" | dedup Group | table Group FullName InheritedFrom | map maxsearches=400 search="search index=corp_mss_ad source=\"*GroupsAndUsers*.csv\" AND Group=\"*$Group$*\" | eval IF=\"$InheritedFrom$\", FN=\"$FullName$\"" | stats values(FN) values(IF) values(SamAccountName) count by Group 

In addition I saw another issue with my Search Head.

I made this query:

index=myindex | fields * | table *

(We use fields * as sometimes the search head is not able to retrieve all fields of the log)

The result of this search give us strange behavior. Indeed, we saw strange fields that are not fields configured as header in our CSV.

It seems that Splunk took some data present in our CSV and used it as a field 😕

This is why I ask you for help.

What is the best practice to perform a query using distributed search?

Do I need to prefix all my queries by the splunk_server name?

splunk_server="My indexer" Index="myindex" | ...

I tried also to copy paste the props.conf from my indexer to the search head, but the result is the same.

Do you have any idea?

Or any logs I can search?

[FrankVl]

First thing that comes to mind would be some authorization.conf mismatch which prevents your search head user from accessing the data. Are you using the same user (and role setup etc.) when searching directly on the indexer?

Regarding the unexpected fields: unless explicitly configured otherwise, Splunk will automatically detect key=value pairs in your data. If you don't want that, you need to configure the respective sourcetype with KV_MODE=none (in props.conf).

[danje57]

This is not an authorization problem.

Users have the same rights on both Searchhead and Indexer.

Regarding the parsed data it is only a CSV file with fields and headers like that:
"Field1","Field2","Field3,"Field4", ...

So normaly special characters or comma which are within the quotes are not a problem for Splunk parser.