Splunk Add-on for VMware hydra gateway port 8008 will not start on DCN

[klowk]

Hi all,

i have configured and started the Splunk Add-on for VMware v 4.0.5 on two heavy forwarders.

Unfortunately will the port 8008 not be available after restarting the Splunk instance. I get everytime following entries in splunkd.log

06-16-2023 16:36:51.103 +0200 ERROR ExecProcessor [23751 ExecProcessor] - message from "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/SA-Hydra/bin/bootstrap_hydra_gateway.py" Traceback (most recent c
all last):
06-16-2023 16:36:51.103 +0200 ERROR ExecProcessor [23751 ExecProcessor] - message from "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/SA-Hydra/bin/bootstrap_hydra_gateway.py" File "/opt/splunk/etc/
apps/SA-Hydra/bin/bootstrap_hydra_gateway.py", line 26, in <module>
06-16-2023 16:36:51.103 +0200 ERROR ExecProcessor [23751 ExecProcessor] - message from "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/SA-Hydra/bin/bootstrap_hydra_gateway.py" port, service_log_le
vel, access_log_level = get_gateway_config(session_key)
06-16-2023 16:36:51.103 +0200 ERROR ExecProcessor [23751 ExecProcessor] - message from "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/SA-Hydra/bin/bootstrap_hydra_gateway.py" File "/opt/splunk/etc/
apps/SA-Hydra/bin/bootstrap_hydra_gateway.py", line 16, in get_gateway_config
06-16-2023 16:36:51.103 +0200 ERROR ExecProcessor [23751 ExecProcessor] - message from "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/SA-Hydra/bin/bootstrap_hydra_gateway.py" stanza = HydraGatewa
yStanza.from_name("gateway", "SA-Hydra", session_key=session_key)
06-16-2023 16:36:51.103 +0200 ERROR ExecProcessor [23751 ExecProcessor] - message from "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/SA-Hydra/bin/bootstrap_hydra_gateway.py" File "/opt/splunk/etc/
apps/SA-Hydra/bin/hydra/models.py", line 610, in from_name
06-16-2023 16:36:51.103 +0200 ERROR ExecProcessor [23751 ExecProcessor] - message from "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/SA-Hydra/bin/bootstrap_hydra_gateway.py" host_path=host_path)
06-16-2023 16:36:51.103 +0200 ERROR ExecProcessor [23751 ExecProcessor] - message from "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/SA-Hydra/bin/bootstrap_hydra_gateway.py" File "/opt/splunk/lib/python3.7/site-packages/splunk/models/base.py", line 552, in get
06-16-2023 16:36:51.103 +0200 ERROR ExecProcessor [23751 ExecProcessor] - message from "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/SA-Hydra/bin/bootstrap_hydra_gateway.py" entity = self._get_entity(id, host_path=host_path)
06-16-2023 16:36:51.103 +0200 ERROR ExecProcessor [23751 ExecProcessor] - message from "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/SA-Hydra/bin/bootstrap_hydra_gateway.py" File "/opt/splunk/etc/apps/SA-Hydra/bin/hydra/models.py", line 339, in _get_entity
06-16-2023 16:36:51.103 +0200 ERROR ExecProcessor [23751 ExecProcessor] - message from "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/SA-Hydra/bin/bootstrap_hydra_gateway.py" splunk.entity.getEntity(self.model.resource, None, sessionKey=self.sessionKey, uri=mid, hostPath=host_path))
06-16-2023 16:36:51.103 +0200 ERROR ExecProcessor [23751 ExecProcessor] - message from "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/SA-Hydra/bin/bootstrap_hydra_gateway.py" File "/opt/splunk/lib/python3.7/site-packages/splunk/entity.py", line 277, in getEntity
06-16-2023 16:36:51.103 +0200 ERROR ExecProcessor [23751 ExecProcessor] - message from "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/SA-Hydra/bin/bootstrap_hydra_gateway.py" serverResponse, serverContent = rest.simpleRequest(uri, getargs=kwargs, sessionKey=sessionKey, raiseAllErrors=True)
06-16-2023 16:36:51.103 +0200 ERROR ExecProcessor [23751 ExecProcessor] - message from "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/SA-Hydra/bin/bootstrap_hydra_gateway.py" File "/opt/splunk/lib/python3.7/site-packages/splunk/rest/__init__.py", line 553, in simpleRequest
06-16-2023 16:36:51.103 +0200 ERROR ExecProcessor [23751 ExecProcessor] - message from "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/SA-Hydra/bin/bootstrap_hydra_gateway.py" raise splunk.AuthenticationFailed
06-16-2023 16:36:51.103 +0200 ERROR ExecProcessor [23751 ExecProcessor] - message from "/opt/splunk/bin/python3.7 /opt/splunk/etc/apps/SA-Hydra/bin/bootstrap_hydra_gateway.py" splunk.AuthenticationFailed: [HTTP 401] Client is not authenticated

Does someone any idea to solve this issue with Authentication failed?

Kind regards

Kathrin

[pemancha]

Please refer to this URL(Troubleshoot the Splunk Add-on for VMware ), it should be able to help resolving the issues that you are having, because I can see in the error message there is splunk.AuthenticationFailed appeared in the error log that you posted.

Check $SPLUNK_HOME/etc/apps/Splunk_TA_vmware/default/inputs.conf

[script://$SPLUNK_HOME/etc/apps/Splunk_TA_vmware/bin/ta_vmware_hierarchy_agent.py]
passAuth = splunkd-user

[klowk · Accepted solution]

yes this was helpful after adding following line the port was open and data was indexed fine .

/opt/splunk/etc/apps/SA-Hydra/local
-bash-4.2$ cat inputs.conf
[script://$SPLUNK_HOME/etc/apps/SA-Hydra/bin/bootstrap_hydra_gateway.py]
disabled = 0
passAuth = <splunk admin user>