Installation

Splunk failed to start after upgrading from V4.3.2 to V5.0.2

tsunamii
Path Finder


[build 149561] 2013-02-15 08:52:24
Received fatal signal 6 (Aborted).
Cause:
Signal sent by PID 15468 running under UID 579.
Crashing thread: MainTailingThread
Registers:
RIP: [0x00007F3A3FB7D8A5] gsignal + 53 (/lib64/libc.so.6)
RDI: [0x0000000000003C6C]
RSI: [0x0000000000003C9B]
RBP: [0x0000000001307210]
RSP: [0x00007F3A35FF6278]
RAX: [0x0000000000000000]
RBX: [0x00007F3A417A5000]
RCX: [0xFFFFFFFFFFFFFFFF]
RDX: [0x0000000000000006]
R8: [0x00007F3A3F400000]
R9: [0x00007F3A4029AEB0]
R10: [0x0000000000000008]
R11: [0x0000000000000206]
R12: [0x0000000001307930]
R13: [0x0000000001306DE0]
R14: [0x00007F3A38438288]
R15: [0x00007F3A384381C8]
EFL: [0x0000000000000206]
TRAPNO: [0x0000000000000000]
ERR: [0x0000000000000000]
CSGSFS: [0x0000000000000033]
OLDMASK: [0x0000000000000000]

OS: Linux
Arch: x86-64

Backtrace:
[0x00007F3A3FB7D8A5] gsignal + 53 (/lib64/libc.so.6)
[0x00007F3A3FB7F085] abort + 373 (/lib64/libc.so.6)
[0x00007F3A3FB76A1E] ? (/lib64/libc.so.6)
[0x00007F3A3FB76AE0] __assert_perror_fail + 0 (/lib64/libc.so.6)
[0x00000000006FCD42] _ZN16FileInputTracker10computeCRCEPm14FileDescriptorRK3Strll + 1906 (splunkd)
[0x00000000006FCE71] _ZN16FileInputTracker11fileHalfMd5EPm14FileDescriptorRK3Strll + 17 (splunkd)
[0x000000000071B844] _ZN3WTF13loadFishStateEb + 644 (splunkd)
[0x000000000070A6C5] _ZN10TailReader8readFileER15WatchedTailFileP11TailWatcher + 149 (splunkd)
[0x000000000070A8E4] _ZN11TailWatcher8readFileER15WatchedTailFile + 260 (splunkd)
[0x000000000070C9FB] _ZN11TailWatcher11fileChangedEP16WatchedFileStateRK7Timeval + 363 (splunkd)
[0x0000000000D3F4E1] _ZN30FilesystemChangeInternalWorker15callFileChangedER7TimevalP16WatchedFileState + 113 (splunkd)
[0x0000000000D40DCF] _ZN30FilesystemChangeInternalWorker12when_expiredERy + 479 (splunkd)
[0x0000000000DA5553] _ZN11TimeoutHeap18runExpiredTimeoutsER7Timeval + 227 (splunkd)
[0x0000000000D3A318] _ZN9EventLoop3runEv + 216 (splunkd)
[0x000000000071328F] _ZN11TailWatcher3runEv + 143 (splunkd)
[0x00000000007133EB] _ZN13TailingThread4mainEv + 267 (splunkd)
[0x0000000000DA2F32] _ZN6Thread8callMainEPv + 66 (splunkd)
[0x00007F3A41373851] ? (/lib64/libpthread.so.0)
[0x00007F3A3FC3311D] clone + 109 (/lib64/libc.so.6)
Linux / saomb040 / 2.6.32-279.11.1.el6.x86_64 / #1 SMP Sat Sep 22 07:10:26 EDT 2012 / x86_64
Last few lines of stderr (may contain info on assertion failure, but also could be old):
splunkd: /opt/splunk/p4/splunk/branches/5.0.2/src/pipeline/input/FileInputTracker.cpp:229: static bool FileInputTracker::computeCRC(uint64_t*, FileDescriptor, const Str&, file_offset_t, file_offset_t): Assertion bytesToHash < 1048576' failed.
2013-02-15 08:49:32.034 -0500 splunkd started (build 149561)
2013-02-15 08:50:53.469 -0500 Interrupt signal received
2013-02-15 08:51:06.551 -0500 splunkd started (build 149561)
2013-02-15 08:52:13.072 -0500 Interrupt signal received
2013-02-15 08:52:23.736 -0500 splunkd started (build 149561)
splunkd: /opt/splunk/p4/splunk/branches/5.0.2/src/pipeline/input/FileInputTracker.cpp:229: static bool FileInputTracker::computeCRC(uint64_t*, FileDescriptor, const Str&, file_offset_t, file_offset_t): Assertion
bytesToHash < 1048576' failed.

/etc/redhat-release: Red Hat Enterprise Linux Server release 6.3 (Santiago)
glibc version: 2.12
glibc release: stable
Threads running: 34
argv: [splunkd -p 9089 start]
terminating...

Splunk appears to start fine, then splunkd fails miserably with a horrible crash creating the above crashlog and the CRC error on bytesToHash

Tags (3)
0 Karma

splunkIT
Splunk Employee
Splunk Employee

We have ran into a similar problem after upgrading to 5.0.2. It seemed to have been caused by one of the apps which was not up-to-dated, and not compatible with the latest splunk version. We started by moving most of the apps (except for the default ones provided during the splunk install) to a temp location, and was able to startup splunk.

Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...