Installation

Why am I getting a server error on login page with splunkd.log showing "reason=bad-cval" after upgrade to 6.2.2?

felipetesta
Path Finder

Just upgraded from 6.1.1 to 6.2.2 (splunk-6.2.2-255606-linux-2.6-x86_64.rpm) on a RHEL 5.7.

At the login page, any valid login/password pair returns a "Server error" red message and splunkd.log reports:

03-09-2015 09:50:00.696 +0100 ERROR UiAuth - user=admin action=login status=failure reason=bad-cval useragent="Mozilla/5.0 (Windows NT 6.1; WOW64; rv:36.0) Gecko/20100101 Firefox/36.0" clientip=10.1.2.3

We have not found previous references to the "bad-cval" error. Typing a wrong password returns an auth failure, therefore it is something happening after the login validation.

What should we look for?

Labels (2)
Tags (3)
1 Solution

felipetesta
Path Finder

Following my last comment, I retouched the configuration.

In $SPLUNK_HOME/etc/system/local/web.conf I had defined this:

root_endpoint = /splunk/

As soon as I edited it into

root_endpoint = /splunk

and restarted ... user access was restored.

Curiously this worked up to 6.1 and broke on 6.2.2, probably with the removal of SplunkWeb process (?).

View solution in original post

fbotte
New Member

had the same issue with splunk 6.5.3
in my case the problem was the firefox extension priv3+

0 Karma

felipetesta
Path Finder

Following my last comment, I retouched the configuration.

In $SPLUNK_HOME/etc/system/local/web.conf I had defined this:

root_endpoint = /splunk/

As soon as I edited it into

root_endpoint = /splunk

and restarted ... user access was restored.

Curiously this worked up to 6.1 and broke on 6.2.2, probably with the removal of SplunkWeb process (?).

felipetesta
Path Finder

Going further with debug, I activated ALL possible debugs and filtering out events I came up with this:

03-09-2015 15:35:10.609 DEBUG HTTPAuthManager - login...
03-09-2015 15:35:10.609 DEBUG AuthenticationManagerSplunk - Attempting login for user 'admin'
03-09-2015 15:35:10.613 DEBUG UserManagerPro - user="admin" successfully logged in
03-09-2015 15:35:10.613 DEBUG HTTPAuthManager - login successful
03-09-2015 15:35:10.613 INFO UserManager - Setting user context: splunk-system-user
03-09-2015 15:35:10.613 INFO UserManager - Done setting user context: NULL -> splunk-system-user
03-09-2015 15:35:10.613 DEBUG AuthenticationManagerSplunk - Getting info for user: admin
03-09-2015 15:35:10.613 DEBUG AuthenticationManagerSplunk - Getting info for user: admin
03-09-2015 15:35:10.613 DEBUG AuthenticationManagerSplunk - Getting info for user: admin
03-09-2015 15:35:10.613 INFO UserManager - Unwound user context: splunk-system-user -> NULL
03-09-2015 15:35:10.613 ERROR UiAuth - user=admin action=login status=failure reason=bad-cval useragent="Mozilla/5.0 (Windows NT 6.1; WOW64; rv:36.0) Gecko/20100101 Firefox/36.0" clientip=10.1.2.3

I doubt I can get more info out of debug logs. If I am wrong, please suggest how to proceed. This is a live, production, Splunk instance with GBs of logs per day, non-stop indexing.

0 Karma

phoffman_splunk
Splunk Employee
Splunk Employee

stand alone "all in one box" or what is your environment like?

LDAP configured or splunk auth?

did you try other browsers?

other users having same issue?

0 Karma

felipetesta
Path Finder

Thank you for jumping in.

It's a all-in-one-box that worked until we upgraded to 6.2.2. We use LDAP auth, but "admin" has always authenticated against the Splunk internal password db. Neither admin nor LDAP accounts can now access Splunk webUI. Tried IE (first time it met Splunk) and FF36 on Win7, with clean cookies: "Server error".

The server is not swapping to disk. I am currently at home and cannot provide other details. Please let me know which other environment info could be useful.

0 Karma

phoffman_splunk
Splunk Employee
Splunk Employee

What is the locale in the URL?

Like mine is "en-US" here "http://searchhead:8000/en-US/app/launcher/home"

0 Karma

felipetesta
Path Finder

When I call "http://searchhead:8000/en-US/app/launcher/home" I get redirected to
"http://searchhead:8000/splunk**//**en-US/account/login?return_to=%2Fsplunk%2Fen-US%2Fapp%2Flauncher%...". Note the double slash before the locale.

If I manually edit the URL and let only one slash then access is possible.

Problem is: when I log out I am thrown again at an URL with double slash, that won't let me in.

We can correct our bookmarks, but it worked with all Splunk versions throughout 6.1.1 and broke on 6.2.2. Is there a config setting to tweak?

This happens with all locales I tried: en-US, en-GB, it-IT.

felipetesta
Path Finder

"cval" is one of the cookies set by Splunk. But even cleaning up the browser doesn't solve the issue. The cookie is regenerated (sample content 657767914).

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...