Why did our Splunk 6.4.x instance on Linux downgrade back to 6.1.3 two weeks after we upgraded?

[sumit9999]

So we upgraded Splunk 6.1.3 to latest 6.4 version. This is a Linux server. Things worked fine for about 2 weeks with new version. This morning, Splunk was down and after bringing it back up looks like it automatically downgraded back to the old version. Trying to understand what might have caused this?

[richgalloway]

AFAIK, Splunk does not automatically downgrade itself. Could someone have restored a system backup over the weekend? Or rolled-back a VM snapshot?

[JDukeSplunk]

Because Splunk is file based, and so is Linux for that matter. The only way that could have happened is if the old Spunk/bin directory still exists on the system somewhere. You may want to run this from the root of your linux box and see if multiple copies of Splunk are installed.

find -name splunkd -perm /a=x

There really should only be one instance of /bin/splunkd

[sumit9999]

yes ... older version of splunk was started which resided in /opt/splunk/bin, whereas new location is : /opt/splunk/splunk/bin . So when I unzipped the upgrade (splunk-6.4.2-00f5bb3fa822-Linux-x86_64.tar) package, I was under impression that it overrides older dirs with the updated changes , is that not the case?

I thank you for your response !

[richgalloway]

Yes, tar will overwrite existing files, but only if you're in the right place (/opt) at the time or use the -C /opt option.