Why am I getting "500 Internal Server Error" when I click "Data Inputs" under "Settings"?

[benjamin009]

We have a fresh Splunk 6.3 install. We literally have 0 data currently indexing. When I click Settings -> Data Inputs. I am getting the error:

500 Internal Server Error
Return to Splunk home page
View more information about your request (request ID = tonsofnumbersandletters) in Search 
This page was linked to from http://x.x.x.x:8000/en-US/app/search/search.
You are logged into x.x.x.x:8000 as admin, which is connected to splunkd @ xxx

When I attempt to search error logs for the specific request ID, nothing populates. There is nothing in any of the error logs referencing this error. I have attempted several searches on Splunk Answers and this seemed to be a common thing for 5.x. I have yet to see a response that works with my issue. I also installed S.o.S (Splunk on Splunk) and do not see anything out of the ordinary that would cause this issue.

Both the search head and Indexer are CentOS 7 running with the user Splunk, not root. I am running out of ideas. Any help is appreciated.

P.S. When I go into Settings - Add Data. I am able to load that properly, but I am not able to view the data inputs.

[michealrp]

It's not just forwarders or changing licenses, I'm running enterprise with a 50gb perpetual, at one time, I had a deployment server set up on my search head, as I only had 2 indexers at the time, since then, I've added a disabled=true in my serverclass.conf file and been going about my business. Now, with the recent update, I've had to remove/rename the serverclass.conf files from local AND default in order to get access to my data inputs section. Otherwise, the system errors out with the 500 internal server error and the logs indicate that the deployment server couldn't be started.

As I don't want to run the deployment server, nor do I really need to run it, I can leave it on, however, I'd rather not have to worry about changing the systems configuration every time an update comes about.

[cpt12tech]

Did you see Pablo's answer? It's a known bug that will be fixed in the 6.4.3 release. Options are to downgrade to 6.4.1 or wait for the release. I'm going to wait for the release as I haven't downgraded before and don't want to risk losing data.

[cpt12tech]

Upgrading to 6.4.3 resolved this for me! I can access the data inputs using the Web GUI.

[ppablo]

Thanks everyone in the community for bringing attention to this issue on Splunk Answers. A bug was filed the last week of July and is tracked as a known issue under SPL-124923: Selecting Data Inputs under Settings results in a "500 Internal Server Error" in Splunk Enterprise 6.4.2 on a Forwarder or Free license.
http://docs.splunk.com/Documentation/Splunk/6.4.2/ReleaseNotes/KnownIssues#Splunk_Web_and_Home_inter...

The fix for this bug is expected in the upcoming 6.4.3 release. The current workaround is to downgrade to 6.4.1 (as some of you have already done) for instances with a Forwarder or Free license.

[cpt12tech]

This thread has been marked answered, I've started another one with the same issue here:
clicking on data inputs shows 500 internal server error

[michealrp]

I don't use the deployment server, and my serverclass.conf file had only the global section setting it to disabled. However, in order to get back into my inputs section, I had to enable the service once again. I did so by renaming the serverclass.conf file in local to something else.

[michealrp]

The things I did..

Enable deployment server by removing the serverclass.conf from my local directory. It was set to globally disable it.
This got me around the internal server error but I couldn't see all of the data input options.

Remove the dbx app (/splunk/etc/apps/dbx) completely.
This allowed me to see all of the data input options but I don't want to use deployment server. Disabling it causes the internal server error.

Falling pack to 6.4.1, with no dbx, no deployment server active, works as it should, at least for me.

[cpt12tech]

We don't have the serverclass.conf in the local directory, so we aren't using it at all. I tried moving a copy from the default folder to the local folder, restarted Splunk, but no change for me.

[smemobox]

Still 500 error for me after applying your suggestion.

[sscarcella]

Yes, it seems that the 6.4.2 update is not compatible with the free version. I backed up and uninstalled then reinstalled 6.4.1 and it worked fine.

[smemobox]

I confirm.
All free servers upgraded to 6.4.2 are suffering data inputs wise

[t00rnad00]

Same problem.
Also loop with Internet Explorer 11 - "Your session has expired. Log in to return to the system."

[brianeh]

I would agree.
I had it working, then as soon as I switched to the Free license again, it stopped working again.
Back tot he 500 error.
I hope there is a fix soon.

[protius2]

I have the exact same issue but I did not turn off splunk_httpinput. I cycled it as disabled and re-enabled it just for grins and restarted Splunk, but this did not help. Any other suggestions?

[benjamin009 · Accepted solution]

For those of you that are interested in the resolution. I (stupidly) disabled the splunk_httpinput app on the indexer. Once I re-enabled it, trying to add data worked just fine.

Dumb mistake by me. Dont be like me.

[epacke]

Plus points to whomever also lets me know how to check if it's disabled, and how to enable it. 🙂

[sscarcella]

HELP... I am using the free version so there is no support. I am getting this error and have not disabled the httpinput. It happened after upgrading to 6.4.2. Getting tons of these in the log file: -0500 WARN IntrospectionGenerator:resource_usage - RU - Failure executing PDH query, skipping getting iostats data this collection cycle. Status code is -2147481643

[epacke]

+1 from me. After upgrading it stopped working.

[brianeh]

This didn't work for me.
This has been new since I upgraded to 6.4.2

[brianeh]

I ended up having to flip from free back to the enterprise free trial in order to be able to open Setting -> Data Inputs again
Due to a logged error that DeployServer was not enabled.
Which was the root of my 500 error.