Deployment Architecture

Deployment client not getting all matching deployment apps

chanfoli
Builder

I have a handful of deployment clients which are not pulling an app which should match the hostname specified in the whitelist, I even tried IP address in whitelist and the client still only pulls a subset of apps which is should deploy. Other clients with similar hostnames work just fine.

Examples from serverclass.conf on DS:

[serverClass:prod_web]
restartSplunkd=true
whitelist.0=ord-*ws*
whitelist.1=ord-*jumpweb*
whitelist.2=pdx-*ws*
whitelist.3=pdx-*jumpweb*
whitelist.4=172.20.18.95
whitelist.5=ord-wl-prod-v6adwws*
blacklist.0=*db*
[serverClass:prod_web:app:ts_prod_apache_inputs]

On incomplete deployment client:

# hostname
pdx-wl-prod-v6adwws02

# ls /opt/splunkforwarder/etc/apps/
custom_deploymentclient   learned  SplunkUniversalForwarder
custom_forwarder_outputs  search   ts_prod_linux_inputs

Note that the apache app is not deployed.

On complete deployment client:

#hostname
pdx-wl-prod-v6adwws01

 # ls /opt/splunkforwarder/etc/apps/
custom_deploymentclient   search                    ts_prod_linux_inputs
custom_forwarder_outputs  SplunkUniversalForwarder
learned                    ts_prod_apache_inputs

apache imputs from prod_web app was deployed 😕

Seems that a random few deployment clients are missing the prod_web app. Reloading restarting deployment server and clients has no impact. The ip address in whitelist.4 value is actually the IP of pdx-wl-prod-v6adwws02 which I added to further troubleshoot. The client still will not pull the app. Any ideas?

0 Karma
1 Solution

kaufmanm
Communicator

Everything in the configs you've posted looks fine. I'd try setting a clientName in deploymentclient.conf and then using that for whitelist.0 of the serverClass. I'd restart the deployment server and then the forwarder. I don't have a great theory as to what is causing the problem. Paste the entire serverclass.conf up to where that ended and it might help with troubleshooting.

View solution in original post

kaufmanm
Communicator

Everything in the configs you've posted looks fine. I'd try setting a clientName in deploymentclient.conf and then using that for whitelist.0 of the serverClass. I'd restart the deployment server and then the forwarder. I don't have a great theory as to what is causing the problem. Paste the entire serverclass.conf up to where that ended and it might help with troubleshooting.

chanfoli
Builder

While I don't like the solution of defining a clientName on the clients which are not matching up to server classes which look correct on the deployment server, the suggestion offered does work. I even defined client name to exactly what hostname should be matching to, after doing this and restarting splunk on the deployment client, the app is deployed as desired.

Thanks,
Sean

0 Karma

chanfoli
Builder

antlefebvre: inputs.conf and server.conf contain the same hostname as the hostname command returns. Thanks for the comment though.

0 Karma

chanfoli
Builder

somesoni2: I see Reload entries every minute for the apps that are deployed on the 02 server, nothing about the missing prod_web app. The servers with the full set of apps show the same events with the addition of "reload" entries for the prod_web app.

0 Karma

antlefebvre
Communicator

hostname from the cli isn't always the same as the hostname of the forwarder. Check your $SPLUNK/etc/system/local/inputs.conf and see what the host= is for that server.

0 Karma

somesoni2
SplunkTrust
SplunkTrust

See if you get any event pertaining to the issue.

index=_internal source=*splunkd.log depl* host=pdx-wl-prod-v6adwws02

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...