Deployment Architecture

distributed search query works (kinda) but only returns single

sdewar83
Path Finder

Hi,

We have 10 sites each with their own splunk server (search head, indexer etc). Each is collecting the same information and has the same index names. I want to run a distributed search queries so that i dont have to log onto each of them and query them individually. I know you can edit the .conf file and create distributed search groups but i'd need to log an RFC for that, so as a proof of concept i just wanted to try and do it using the splunk_server= command. If i choose a search that works fine one search head and add in some logic to try and send it to multiple search heads, it seems to return a single result and I can't seem to get it to show multiple figures.

e.g i'm trying stuff like:

index=* OR index=_* AND splunk_server=yyyyyyyyyyyyy OR splunk server=xxxxxxxxxxxxxxxxx
| fields, sourcetype, _raw
| eval size-len(_raw)
|stats sum(size) as size
| eval size=round(size/1024/1024,2)

but no joy? i'd have hoped it'd show the MB size of raw data capture by the servers at both sites. I think it only shows yyyyyyyyyyyy.

p.s also if i piped it to a table, what field would i have to use to display which search head the respective results came from?

Many thanks,

Tags (1)
0 Karma

adonio
Ultra Champion

try this, what are the results?

index=* OR index=_*  (splunk_server=yyyyyyyyyyyyy OR splunk_server=xxxxxxxxxxxxxxxxx)
| fields, sourcetype, _raw
| eval size=len(_raw)
|stats sum(size) as size by splunk_server
| eval size=round(size/1024/1024,2)
0 Karma

sdewar83
Path Finder

Hmmmn.

I tried your suggestion and it came up with 0 events. I tried using FQDNs for the server names, no joy. Tried FQDN:port, no joy. No joy either for IP or IP:port. Splunk_Server=* seems to work. (p.s is the port the same port number thats in the web console url or is it 8089? i tried both, no joy)

i can't even get it to work at all now. not sure what's changed. I can't even get splunk_server=local to return a result. Either i dont use the command and the search runs as normal or i use splunk_server=*.

0 Karma

adonio
Ultra Champion

i missed an underscore _ in my search, and fixed it

when you are searching this:

index=_internal  splunk_server=*
 | fields, sourcetype, _raw
 | eval size=len(_raw)

do you see the field size ?

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...