Deployment Architecture

How to troubleshoot why I am unable to start Splunk Web using version 6.2.2 on RHEL 6.5?

mmohiuddin
Path Finder

Hi:

I am unable to start splunk web on version 6.2.2 on RHEL 6.5.

Checking prerequisites...
        Checking http port [8001]: open
        Checking mgmt port [8089]: open
The Splunk web interface is at https://awspoclts21:8001

I disabled the local firewall, but in vain. Still unable to start Splunk Web. Is there something I am missing. There is nothing on the web_services.log

015-04-07 11:12:31,979 INFO    [5523f3df637fb21415ec90] root:635 - CONFIG: version_label (str): 6.2.2
2015-04-07 11:12:31,979 INFO    [5523f3df637fb21415ec90] root:635 - CONFIG: version_number (str): 6.2.2
2015-04-07 11:12:31,979 INFO    [5523f3df637fb21415ec90] root:635 - CONFIG: x_frame_options_sameorigin (bool): True
2015-04-07 11:12:31,980 INFO    [5523f3df637fb21415ec90] root:693 - DJANGO: configuring...
2015-04-07 11:12:32,122 INFO    [5523f3df637fb21415ec90] root:735 - DJANGO: not starting, found no apps
2015-04-07 11:12:32,122 INFO    [5523f3df637fb21415ec90] root:129 - ENGINE: Bus STARTING
2015-04-07 11:12:32,132 INFO    [5523f3df637fb21415ec90] root:129 - ENGINE: Started monitor thread '_TimeoutMonitor'.
2015-04-07 11:12:32,239 INFO    [5523f3df637fb21415ec90] root:129 - ENGINE: Serving on 127.0.0.1:8065
2015-04-07 11:12:32,240 INFO    [5523f3df637fb21415ec90] root:129 - ENGINE: Bus STARTED
2015-04-07 11:12:32,286 INFO    [5523f3e0487fb211aa1210] root:129 - ENGINE: Started monitor thread 'Monitor'.

splunkd.log

04-07-2015 11:13:01.406 -0400 INFO  KeyManagerLocalhost - Finished reading public key for localhost: /opt/splunk/etc/auth/distServerKeys/trusted.pem
04-07-2015 11:13:01.406 -0400 INFO  KeyManagerLocalhost - Reading private key for localhost: /opt/splunk/etc/auth/distServerKeys/private.pem
04-07-2015 11:13:01.406 -0400 INFO  KeyManagerLocalhost - Finished reading private key for localhost: /opt/splunk/etc/auth/distServerKeys/private.pem
04-07-2015 11:13:21.124 -0400 WARN  TcpOutputProc - Cooked connection to ip=10.143.28.20:9997 timed out
04-07-2015 11:13:43.126 -0400 INFO  HttpPubSubConnection - Running phone uri=/services/broker/phonehome/connection_10.143.29.20_8089_awspoclts21.rsc.humad.com_awspoclts21_0A653A89-0248-4601-A642-A716953F5A43
04-07-2015 11:13:43.225 -0400 INFO  HttpPubSubConnection - Running phone uri=/services/broker/phonehome/connection_10.143.29.20_8089_awspoclts21.rsc.humad.com_awspoclts21_0A653A89-0248-4601-A642-A716953F5A43
04-07-2015 11:13:51.125 -0400 WARN  TcpOutputProc - Cooked connection to ip=10.143.28.20:9997 timed out
04-07-2015 11:14:21.127 -0400 WARN  TcpOutputProc - Cooked connection to ip=10.143.28.20:9997 timed out
04-07-2015 11:14:43.252 -0400 INFO  HttpPubSubConnection - Running phone uri=/services/broker/phonehome/connection_10.143.29.20_8089_awspoclts21.rsc.humad.com_awspoclts21_0A653A89-0248-4601-A642-A716953F5A43
04-07-2015 11:14:51.128 -0400 WARN  TcpOutputProc - Cooked connection to ip=10.143.28.20:9997 timed out
04-07-2015 11:15:21.130 -0400 WARN  TcpOutputProc - Cooked connection to ip=10.143.28.20:9997 timed out

We can ignore the errors on splunkd.log as this is a fresh install and I am planning to add port 9997 through the web UI

0 Karma

sraarum
New Member

Try accessing localhost:8001, localhost:8002. I had similar issue and starting the splunk process clarified the logs. Which started the web UI on port 8002
,Try accessing localhost:8001, localhost:8002.
I had similar issue and the splunk start process clarified the web UI was started on port 8002.

0 Karma

mmohiuddin
Path Finder

Yes I am going to awspoclts21:8001. I see using netstat command that the port 8001 is listening. But when I am doing a telnet from my local host pc onto awspoclts21 on port 8000 it is stating connection refused. I spoke with the network firewall team and they also suggest they don't have any firewall rules set on port 8000. I need to check on the server side.

On the server side I have disabled the firewall services, using service iptables stop.

Could you suggest how do I proceed next?

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

You have configured splunkweb to run on port 8001, then why are you checking port 8000 on firewall? Telnet from your machine to awspoclts21 on port 8001

0 Karma

mmohiuddin
Path Finder

I have changed the port to 8001 and telnet doesn't work. First tried changing web port from 8000 to 8001 and did telnet from my machine to awspoclts21 on port 8001. That doesn't work.

0 Karma

harsmarvania57
SplunkTrust
SplunkTrust

Then port 8000 & 8001 are blocked from your local machine to awspoclts21. Check with your network admin that any firewall rule is blocking this traffic.

0 Karma

fdi01
Motivator

to start Splunk Web, go to browser and run:

http://localhost:8001
if you wan u can run splunkd to do it go to Splunk_home/bin directory and run splunk start command
before run in browser:
http://localhost:8001
or
127.0.0.1:8001

0 Karma

rsennett_splunk
Splunk Employee
Splunk Employee

As per your first bits it is started... do you mean you can't access it?

I guess I have to ask... it looks like you changed the default port for the web interface. And you're sure you're going to https://awspoclts21:8001 and not https://awspoclts21:8000 out of habit? And you tried using the IP address rather than the hostname, yes?

Also... you've edited the startup feedback quite a bit. You might want to post the whole thing... Just in case there is something odd and easy to miss.

With Splunk... the answer is always "YES!". It just might require more regex than you're prepared for!
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...