Hello all !
Environment:
- OS - Ubuntu 12.04.6
- Splunk - splunk-6.1.3-220630-linux-2.6-amd64.deb
- Installed via dpkg
- Independent installation
I'm having an issue with starting splunkweb on the first go.
Here is the error from web_services.log
2014-10-03 19:17:09,447 INFO [542ef6340610d2450] root:138 - ENGINE: Bus STARTING
2014-10-03 19:17:09,475 INFO [542ef6340610d2450] root:138 - ENGINE: Started monitor thread '_TimeoutMonitor'.
2014-10-03 19:17:09,492 ERROR [542ef6340610d2450] root:138 - ENGINE: Error in HTTP server: shutting down
Traceback (most recent call last):
File "/opt/splunk/lib/python2.7/site-packages/cherrypy/process/servers.py", line 73, in _start_http_thread
self.httpserver.start()
File "/opt/splunk/lib/python2.7/site-packages/cherrypy/wsgiserver/__init__.py", line 1725, in start
raise socket.error, msg
error: [Errno 13] Permission denied
2014-10-03 19:17:09,492 INFO [542ef6340610d2450] root:138 - ENGINE: Bus STOPPING
2014-10-03 19:17:09,492 ERROR [542ef6340610d2450] root:138 - ENGINE: Error in 'stop' listener <bound method Server.stop of <cherrypy._cpserver.Server object at 0x10d2350>>
Traceback (most recent call last):
File "/opt/splunk/lib/python2.7/site-packages/cherrypy/process/wspbus.py", line 147, in publish
output.append(listener(*args, **kwargs))
File "/opt/splunk/lib/python2.7/site-packages/cherrypy/process/servers.py", line 107, in stop
self.httpserver.stop()
File "/opt/splunk/lib/python2.7/site-packages/cherrypy/wsgiserver/__init__.py", line 1909, in stop
self.requests.stop(self.shutdown_timeout)
File "/opt/splunk/lib/python2.7/site-packages/cherrypy/wsgiserver/__init__.py", line 1458, in stop
self._monitor_thread.join()
AttributeError: 'NoneType' object has no attribute 'join'
2014-10-03 19:17:09,494 INFO [542ef6340610d2450] root:138 - ENGINE: Stopped thread '_TimeoutMonitor'.
To replicate:
Any time I restart splunk (/etc/init.d/splunk restart
OR $SPLUNK_HOME/bin/splunk restart
) the splunkd restarts fine but web_services.log throws the error above.
If I follow the restart with a splunk start splunkweb
the web server comes up with no errors. Main problem with this is that on system start, web server does not fire off and I have to manually log in to start it.
Tried:
- Reinstalling package
- Changing permissions
- Changed to root (to test)
- Also verified port is not in use before starting (shouldn't matter on restart)
Anyone encounter this issue ?
Looks to me like you previously started Splunk as root (started on boot?), and now you are trying to start it as some unpriveleged user (perhaps the splunk
user itself) as a manual start.
When I added it to the startup scripts I tried both combinations (as root user and as splunk).
Regardless of what I did both scenarios yielded the same outcome.
I believe that if it was a permissions issue, and I were to run splunk start splunkweb
as root, it would start as expected. No matter which user I use, the first time I run splunk start splunkweb
it renders in the error above. As I explained above, if this is followed by the same command immediately after, the service starts correctly.