web.conf missing & splunkweb doesnt show up in splunk status

[celina11]

Hi: I recently installed Splunk 8.0 on CentOS 8. Post successful installation I am unable to launch Splunk web interface. Upon checking the status - ./splunk status - the output only shows splunkd and splunk helpers and nothing about splunkweb. Further checked for web.conf file in /opt/splunk/etc/system/local/ but the file itself is missing. Do I have to re-install my installation ? Please help.

[chinmoya]

Hi,

Can you run the below command. it might help you in getting the port on which splunk web is running

./splunk show web-port

[celina11]

Hi @chinmoya : when ran the command while the splunkd was not running got the following output -
[root@localhost bin]# ./splunk show web-port
splunkd 8626 was not running.
Stopping splunk helpers...
[ OK ]
Done.
Stopped helpers.
Removing stale pid file... done.
Web port: 8000

With splunkd running, same command prompted for Splunk username & password. When the default credentials were entered admin/changeme system prompted with login failed error
[root@localhost bin]# ./splunk show web-port
Splunk username: admin
Password:
Login failed

[woodcock]

Check your web_service.log and at the very bottom it will identify the app that is causing your problem, something like this:

File "C:\Program Files\Splunk\etc\apps\<app_that_is_causing_the_problem_here>\appserver\modules\CustomRESTForSavedSearch\CustomRESTForSavedSearch.py", line 24
     except Exception, e:
                     ^
 SyntaxError: invalid syntax

The quick solution is to go to the CLI on your Search head, and create a $SPLUNK_HOME/etc/apps/<app_that_is_causing_the_problem_here>/local/app.conf file with the contents below to disable it temporarily and then restart. Splunk will come up and you can then upgrade to the latest version of app_that_is_causing_the_problem_here, live without it, or work with the author to fix the problem if it still exists on the latest version:

[install]
state = disabled

To avoid some of these kinds of problems in the future, be sure to run the Splunk Platform Upgrade Readiness App app before upgrading:
https://splunkbase.splunk.com/app/4698/

[celina11]

Thanks @woodcock for sharing your inputs. Did check and found no errors logged in web_service.log

[ashajambagi]

are you getting any error in splunkd.log?

[celina11]

Hi @ashajambagi : These were the only errors seen in splunkd.log -

01-20-2020 08:33:05.655 +0530 ERROR ProcessRunner - helper process seems to have died (child killed by signal 15: Terminated)!
01-20-2020 08:33:05.655 +0530 WARN EventLoop - ProcessRunner: about to throw an EventLoopException: error from PolledSocket write: Broken pipe
01-20-2020 08:33:05.656 +0530 ERROR ProcessRunner - got exception while running ProcessRunner's EventLoop: ProcessRunner: about to throw an EventLoopException: error from PolledSocket write: Broken pipe

[soumyasaha25]

Hi @celina11 ideally the web.conf file will be locate in /opt/splunk/etc/system/default/ unless you have made some custom changes to it and placed it in /opt/splunk/etc/system/local/.
i did once encounter similar issue in the past (when i had installed splunk as a test on my personal laptop), turned out instead of port 8000 splunk we was setup to use 8002. I would suggest you have a look at the we.conf file and check for the attribute
"httpport" also verify if attribute "startwebserver" is set to 1.
so if the httpport is set to port 8003 (assuming) you should try to access splunk web by hitting http://localhost:8003

Lastly, if you care making any changes to the web.conf file, make sure you create a new web.conf file in /opt/splunk/etc/system/local/ and make your changes there.

[celina11]

Thanks for your reply @soumyasaha25. Was finally able to locate web.conf which is under /opt/splunk/etc/system/default/ . Absolutely new to splunk so learning the basics 🙂 httpport is set as 8000 and startwebserver is 1 in web.conf.

Here's the output when started the splunk process today if this helps -
[root@localhost bin]# ./splunk start

Splunk> CSI: Logfiles.
Checking prerequisites...
Checking http port [8000]: open
Checking mgmt port [8089]: open
Checking appserver port [127.0.0.1:8065]: open
Checking kvstore port [8191]: open
Checking configuration... Done.
Checking critical directories... Done
Checking indexes...
Validated: _audit _internal _introspection _metrics _telemetry _thefishbucket history main summary
Done
Checking filesystem compatibility... Done
Checking conf files for problems...
Done
Checking default conf files for edits...
Validating installed files against hashes from '/opt/splunk/splunk-8.0.1-6db836e2fb9e-linux-2.6-x86_64-manifest'
All installed files intact.
Done
All preliminary checks passed.
Starting splunk server daemon (splunkd)...
Done
[ OK ]

Waiting for web server at http://127.0.0.1:8000 to be available..... Done
If you get stuck, we're here to help.
Look for answers here: http://docs.splunk.com
The Splunk web interface is at http://192.168.122.1:8000

[root@localhost bin]# ./splunk status
splunkd is running (PID: 8626).
splunk helpers are running (PIDs: 8633 8646 8723 8758 8983).

[Javoraqa]

Hi @celina11 ,

Have you created a new web.conf file in local path ?