All Apps and Add-ons

Splunk DB Connect 2 RPC Service not responding

christophe_clem
Explorer

Hello,

I have an issue with Splunk DB Connect 2 on Splunk 6.3.0, running on Red Hat Enterprise Linux 6.6 x64 : the RPC service seems to be running but is very (very very) slow and almost each request ends up with a timeout.

In the Settings page, I got this message after some seconds:

Splunkd daemon is not responding: ("Error connecting to /servicesNS/nobody/splunk_app_db_connect/db_connect/rpcserver/default?output_mode=json: ('The read operation timed out',)",) 

In the logs, there is the same message, but no other error.

I tried Oracle JRE and openJDK, but the result is the same.
I tried Oracle JRE 8u60 and 8u31. I would like to try JRE i586 but the installation of the RPM fails.

The server has 8 CPU cores and 4 GB of memory.

Kind regards,
Christophe

0 Karma

acr
Engager

Same issue, did you find a solution ?

0 Karma

sgundeti
Path Finder

RPC server runs on port 9998 by default, make sure its not used by any other service. If it is being used by some other service, then edit inputs.conf file in dbx app to change default port to unused port and restart splunk.

0 Karma

christophe_clem
Explorer

Nope, nothing else seems to be listening (or try to) on port TCP/9998.

0 Karma

bevant
Explorer

Have you looked in rpc.log and - more likely - dbx2.log as well?

I assume this happens after you've configured it, so what does the dbx app config look like (inputs.conf)

0 Karma

christophe_clem
Explorer

This is right after I configure it, even the setup page is not working after configuration !

The file inputs.conf contains this:

[rpcstart://default]
javahome = /usr/java/latest/
options = -Xmx1024m -XX:+UseConcMarkSweepGC
useSSL = 0
proc_pid = 3839
Exception = 0

dbx2.log:

10/21/2015 17:03:51 [ERROR] [__init__.py] Socket error communicating with splunkd (error=('The read operation timed out',)), path = /servicesNS/admin/splunk_app_db_connect/db_connect/rpcserver
10/21/2015 17:03:51 [ERROR] [__init__.py] Splunkd daemon is not responding: ("Error connecting to /servicesNS/admin/splunk_app_db_connect/db_connect/rpcserver: ('The read operation timed out',)",)
Traceback (most recent call last):
  File "/local/opt/splunk/lib/python2.7/site-packages/splunk/rest/__init__.py", line 185, in dispatch
    methodOutput = method()
  File "/local/opt/splunk/etc/apps/splunk_app_db_connect/bin/dbinfo.py", line 31, in handle_GET
    self.dbx2 = service.build_service(self.get_rpc_channel())
  File "/local/opt/splunk/etc/apps/splunk_app_db_connect/bin/dbinfo.py", line 19, in get_rpc_channel
    rpc = utils.get_local_rpc(self.userName, self.sessionKey)
  File "/local/opt/splunk/etc/apps/splunk_app_db_connect/bin/dbx2/utils.py", line 170, in get_local_rpc
    ent = en.getEntities(["db_connect", "rpcserver"], namespace=namespace, owner=user, sessionKey=sessionKey)
  File "/local/opt/splunk/lib/python2.7/site-packages/splunk/entity.py", line 129, in getEntities
    atomFeed = _getEntitiesAtomFeed(entityPath, namespace, owner, search, count, offset, sort_key, sort_dir, sessionKey, uri, hostPath, **kwargs)
  File "/local/opt/splunk/lib/python2.7/site-packages/splunk/entity.py", line 222, in _getEntitiesAtomFeed
    serverResponse, serverContent = rest.simpleRequest(uri, getargs=kwargs, sessionKey=sessionKey, raiseAllErrors=True)
  File "/local/opt/splunk/lib/python2.7/site-packages/splunk/rest/__init__.py", line 479, in simpleRequest
    raise splunk.SplunkdConnectionException, 'Error connecting to %s: %s' % (path, str(e))
SplunkdConnectionException: Splunkd daemon is not responding: ("Error connecting to /servicesNS/admin/splunk_app_db_connect/db_connect/rpcserver: ('The read operation timed out',)",)
10/21/2015 17:03:51 [ERROR] [__init__.py] Socket error communicating with splunkd (error=('The read operation timed out',)), path = /servicesNS/admin/splunk_app_db_connect/db_connect/rpcserver
10/21/2015 17:03:51 [ERROR] [__init__.py] Splunkd daemon is not responding: ("Error connecting to /servicesNS/admin/splunk_app_db_connect/db_connect/rpcserver: ('The read operation timed out',)",)
Traceback (most recent call last):
  File "/local/opt/splunk/lib/python2.7/site-packages/splunk/rest/__init__.py", line 185, in dispatch
    methodOutput = method()
  File "/local/opt/splunk/etc/apps/splunk_app_db_connect/bin/dbinfo.py", line 31, in handle_GET
    self.dbx2 = service.build_service(self.get_rpc_channel())
  File "/local/opt/splunk/etc/apps/splunk_app_db_connect/bin/dbinfo.py", line 19, in get_rpc_channel
    rpc = utils.get_local_rpc(self.userName, self.sessionKey)
  File "/local/opt/splunk/etc/apps/splunk_app_db_connect/bin/dbx2/utils.py", line 170, in get_local_rpc
    ent = en.getEntities(["db_connect", "rpcserver"], namespace=namespace, owner=user, sessionKey=sessionKey)
  File "/local/opt/splunk/lib/python2.7/site-packages/splunk/entity.py", line 129, in getEntities
    atomFeed = _getEntitiesAtomFeed(entityPath, namespace, owner, search, count, offset, sort_key, sort_dir, sessionKey, uri, hostPath, **kwargs)
  File "/local/opt/splunk/lib/python2.7/site-packages/splunk/entity.py", line 222, in _getEntitiesAtomFeed
    serverResponse, serverContent = rest.simpleRequest(uri, getargs=kwargs, sessionKey=sessionKey, raiseAllErrors=True)
  File "/local/opt/splunk/lib/python2.7/site-packages/splunk/rest/__init__.py", line 479, in simpleRequest
    raise splunk.SplunkdConnectionException, 'Error connecting to %s: %s' % (path, str(e))
SplunkdConnectionException: Splunkd daemon is not responding: ("Error connecting to /servicesNS/admin/splunk_app_db_connect/db_connect/rpcserver: ('The read operation timed out',)",)
10/21/2015 17:03:52 [ERROR] [__init__.py] Socket error communicating with splunkd (error=('The read operation timed out',)), path = /servicesNS/nobody/splunk_app_db_connect/db_connect/rpcserver

rpc.log:

2015-10-21 17:03:37 DEBUG ServiceResponder:396 - Request name: getServerStatus.
2015-10-21 17:03:37 DEBUG ServiceResponder:405 - Request parameter(0): class [Ljava.lang.String; = [[HOSTNAME, HOME, SPLUNK_WEB_NAME, SPLUNK_HOME, SPLUNK_SERVER_NAME, TERM_SESSION_ID]].
2015-10-21 17:03:47 DEBUG ServiceResponder:396 - Request name: getServerStatus.
2015-10-21 17:03:47 DEBUG ServiceResponder:405 - Request parameter(0): class [Ljava.lang.String; = [[SPLUNK_HOME]].
2015-10-21 17:03:57 DEBUG ServiceResponder:396 - Request name: getServerStatus.
2015-10-21 17:03:57 DEBUG ServiceResponder:405 - Request parameter(0): class [Ljava.lang.String; = [[HOSTNAME, HOME, SPLUNK_WEB_NAME, SPLUNK_HOME, SPLUNK_SERVER_NAME, TERM_SESSION_ID]].
2015-10-21 17:04:03 DEBUG ServiceResponder:396 - Request name: getServerStatus.
2015-10-21 17:04:03 DEBUG ServiceResponder:405 - Request parameter(0): class [Ljava.lang.String; = [[SPLUNK_HOME]].
0 Karma

hecorbett
New Member

I'm having similar issues. Did you ever get this resolved?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...