Deployment Architecture

Is there a way to limit the number of clients a Splunk deployment server pushes a new/updated app to at the same time?

jbresciani
New Member

Is there a way to limit the number of clients a Splunk deployment server pushes a new/updated app to at the same time? We've run into issues where to the Splunk universal forwarder is restarted on hundreds of virtual machines simultaneously causing massive IO delays on shared storage.

I'm looking for a way to have a Splunk deployment server stage out the deployment of app changes so that only X number of nodes are deployed to at a time.

0 Karma

esix_splunk
Splunk Employee
Splunk Employee

There is no way to do this with Splunk native functionality aside from groups in serverclasses [ group by subnet / server location / server role / etc.] The best option is to adjust the phonehomeinterval to a level that reduces your i/o to acceptable levels. The caveat to this is that it takes longer for your clients to update.

Outside of Splunk, there are some other things you could potentially do, although not recommended. One option Ive seen is to run with multiple dns cnames for your deployment server. Push your updates in a staggered manny by dropping the cnames out while pushing updates. The UF will still phone home, but wont be able to connect until you put the cnames back in. You need to be aware of the ttl values in dns in this case though and have it set really low.

Another option would be firewall rules, block inbound to the DS for portions of your UF's subnets while updates are being pushed...

sobrien
Splunk Employee
Splunk Employee

Hi Jacob,

A good way to achieve this would be to break up your fleet into smaller serverclasses:

https://docs.splunk.com/Splexicon:Serverclass

You can then confirm the successful install of your apps and updates per serverclass prior to rolling out to the rest of the fleet.

Hope this helps.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...