Could someone give me solution for the problem.
When i am trying to login into splunk showing below message in page.
*500 Internal Server Error
Return to Splunk home page
ResponseNotReady
View more information about your request (request ID = 54219c3173e66d493160) in Search
You are using myHost:8000, which is connected to splunkd @000 at https://127.0.0.1:8089 on Tue Sep 23 11:13:39 2014.*
Thanks for all the info here. I had a similar issue in our lab env. Figured out it was due to mgmt port being used by some other process.
Splunk start command showed me the error and also allowed me to change it.
I'm getting the same issue.
background:
I am automating a base deployment with ansible. The architecture consists of 3 clustered indexers, 3 clustered search heads, 1 cluster master, 1 deployment server, 1 license master. I was able to login without ssl configured. I then edited the web.conf file to enable web ssl and pointed it at the private key and server certificate. I then wiped the servers and re-ran the ansible scripts. I also have splunk running through systemd. I have set the permissions of /opt/splunk to :splunk and made the necessary additions to sudoers.
Result:
The web UI shows up at the set url registering a valid ssl certificate; however, once I utilize the user to login it fails with a 500 Internal Server Error
logs:
The web_service.log has the following entry:
"File "/opt/splunk/lib/python2.7/site-packages/splunk/rest/init.py", line 500, in simpleRequest raise splunk.SplunkdConnectionException, 'Error connecting to %s: %s' % (path, str(e)) SplunkdConnectionException: Splunkd Daemon is not responding: ("Error connecting to /services/authentication/users/: (')ssl.c:725: The handshake operation timed out',)",)"
@jingisonsplunk Did you find a solution to this?
I'm still getting a 500 error on login to the management interface at http://mypublicip:8000/
I've tried all the above suggestions but no joy.
This is running on CentOS 7.5.1804 as the user "splunk" with a home directory of /opt/splunk
Any other suggestions would be greatly appreciated.
,I'm having the same problem - setting:
[httpServer]
disableDefaultPort = false
in /opt/splunk/etc/system/local/server.conf does not fix this. Still getting a 500 error when logging into the admin page at http://mypublicip:8000
I've tried the other recommended settings - namely setting PATH and SPLUNK_HOME vars and sourcing $SPLUNK_HOME/bin/setSplunkEnv
Still no joy - any thoughts?
not sure why the above answer got down votes but web_service.log was the file where I see actual errors regarding the 500 error on the web UI.
This happens when you accidentally turn off the management interface. You can look for this with either of these 2 commands:
find /opt/splunk/etc/ -type f -name server.conf -exec grep -il disableDefaultPort {} \;
/opt/splunk/bin/splunk btool server list --debug | grep disableDefaultPort
To brute force a quick-fix until you sort out your configuration files, just put this in /opt/splunk/etc/system/local/server.conf
:
[httpServer]
disableDefaultPort = false
Then restart Splunk.
To add insult to injury, neither the splunk logs, nor the dead page served to you give you any indication that this is the situation and either could and BOTH SHOULD. Even when we turned on debug with /opt/splunk/bin/splunk start --debug
, we STILL do not get any log telling us that this setting has explicitly disabled this core function. The ONLY place that you see this, and the only reason that we figured it out, is that it IS logged to STDOUT when you start splunk. You will see this somewhat casual note:
$ /opt/splunk/bin/splunk start
Splunk> All batbelt. No tights.
Checking prerequisites...
Management port has been set disabled; the web UI cannot work.
Checking http port [8000]: open
Management port has been set disabled; cli support for this configuration is currently incomplete.
I opened a P4/ER to have this logged as a WARN but who knows if this will ever get implemented. Hopefully this answer will save somebody the day that I wasted on this. To be fair, it was my own fault; I was hardening UFs and did not have my blacklist correct for my server class so it hit a few of my Search Heads. DOH!
Same problem here. UF configs were applied to our HF. Still a good answer 4yrs later. Thank you.
Thank you so much. This and rectifying permissions seemed to fix my issue.
We got this on a brand new 6.6.7 install on RHEL 6.9. The UI was responding very slowly then timing out and giving us the 500 error.
We reinstalled Splunk and evidently at first it was working fine. In the morning, it stopped responding again, so we stopped SSL. Things worked fine, so we turned SSL back on and everything has worked fine since.
Another way this error will occur is if the file ownership or permissions on your Splunk directories are changed.
I had an admin inadvertently move ownership of /splunk on my servers and it immediately started throwing these 500 errors to my splunk users.
A simple 'ls -lar' or splunk status will confirm. Once ownership was restored the 500 errors went away.
Some search and alerting functions did require a restart of splunk to fix.
Thank you. This fixed my issue.
It seems to be happening due to not unsetting proxy values as "akhtet" said.
I used a below generic bash script on my mac and it fixed the problem.
==splunk.sh==
#!/bin/bash
if [ "$1" = "start" ] ; then
export SPLUNK_HOME=/Applications/Splunk/
source $SPLUNK_HOME/bin/setSplunkEnv
export PATH=$SPLUNK_HOME/bin:$PATH
unset http_proxy https_proxy HTTP_PROXY HTTPS_PROXY
fi
/Applications/Splunk/bin/splunk $1
usage :
./splunk.sh start
./splunk.sh stop
./splunk.sh status
./splunk.sh
You may also want to check your proxy settings.
I had the same error message as soon as I login to Splunk.
When I looked into web_service.log, I saw the following error message.
startup:96 - Unable to read in product version information; (403, 'Forbidden')
The root cause is that I have set the http(s)_proxy environment variable (as I'm inside a proxied environment) and Splunk seem to have issues connecting to that proxy to read in product version info. After I have reset the http(s)_proxy variable to empty strings, the issue no longer exists.
Be sure to set the following before starting for the first time:
export SPLUNK_HOME=/my/splunk/dir
source $SPLUNK_HOME/bin/setSplunkEnv
Add $SPLUNK_HOME/bin to your PATH
export PATH=$SPLUNK_HOME/bin:$PATH
Check to make sure 'which python' is referencing the local, splunk, python libs:
[splunkysploo]# which python
/<$SPLUNK_HOME>/bin/python
Then, start Splunk with $SPLUNK_HOME/bin/splunk start as per usual.
This is a very generic error message. The first place I would look would be /splunk/var/log/splunk/web_service.log for further details.