About mdegann

mdegann · ‎04-08-2014

Whenever i try to do a spark line with a certain amount of data the thread crashes and the search doesn't finish. I'm using Splunk 6.0. [build 182037] 2014-04-08 17:40:35 Received fatal signal 11 (Segmentation fault). Cause: No memory mapped at address [0x00007F05D54F2F40]. Crashing thread: dispatch Registers: RIP: [0x00007F051C0E6BC6] free + 598 (/opt/splunk/lib/libjemalloc.so.1) RDI: [0x00007F0517806000] RSI: [0x0000000000000000] RBP: [0x00007F0517808008] RSP: [0x00007F05183FC6F0] RAX: [0x00007F05183FF6E0] RBX: [0x00007F05175B7100] RCX: [0x00007F051C129FB0] RDX: [0x0000000017B37701] R8: [0x0000000000000000] R9: [0x00007F051C129F60] R10: [0x00007F051A800000] R11: [0x00007F0517B37740] R12: [0x0000000000002000] R13: [0x00007F0517806000] R14: [0x00000000000000DE] R15: [0x0000000017B37700] EFL: [0x0000000000010206] TRAPNO: [0x000000000000000E] ERR: [0x0000000000000006] CSGSFS: [0x0000000000000033] OLDMASK: [0x0000000000000000] OS: Linux Arch: x86-64 Backtrace: [0x00007F051C0E6BC6] free + 598 (/opt/splunk/lib/libjemalloc.so.1) [0x000000000105888D] _ZNSt8_Rb_treeI3StrSt4pairIKS0_16StatsInclOptionsESt10_Select1stIS4_ESt4lessIS0_ESaIS4_EE8_M_eraseEPSt13_Rb_tree_nodeIS4_E + 77 ([splunkd) [0x00000000010588F5] _ZN13ScopedPointerISt3mapI3Str16StatsInclOptionsSt4lessIS1_ESaISt4pairIKS1_S2_EEEED2Ev + 21 ([splunkd) [0x000000000104753A] _ZNK14StatsProcessor15output_internalEmR13SearchResultsR17SearchResultsInfobbb + 330 ([splunkd) [0x0000000001056E14] _ZN14StatsProcessor15reduce_and_emitER13SearchResultsR17SearchResultsInfob + 1956 ([splunkd) [0x0000000000FF2A72] _ZN17NewChartProcessor14execute_outputER13SearchResultsR17SearchResultsInfo + 2530 ([splunkd) [0x0000000000FFBE55] _ZN17NewChartProcessor31execute_output_and_set_metadataER13SearchResultsR17SearchResultsInfo + 69 ([splunkd) [0x0000000000F00DE3] _ZN14DispatchThread15generatePreviewERK8Pathname + 3347 ([splunkd) [0x0000000000F013EF] _ZN14DispatchThread15generatePreviewERK8PathnameRK3Str + 47 ([splunkd) [0x0000000000F04218] _ZN14DispatchThread8mainImplEv + 11656 ([splunkd) [0x0000000000F0F0EF] _ZN14DispatchThread4mainEv + 223 ([splunkd) [0x0000000000D81A2D] _ZN6Thread8callMainEPv + 61 ([splunkd) [0x00007F051B132E9A] ? (/lib/x86_64-linux-gnu/libpthread.so.0) [0x00007F051AE5ECCD] clone + 109 (/lib/x86_64-linux-gnu/libc.so.6) Linux / splunk01 / 3.5.0-36-generic / #57-Ubuntu SMP Wed Jun 19 15:10:49 UTC 2013 / x86_64 /etc/debian_version: wheezy/sid MAP: 00400000-017a5000 r-xp 00000000 fc:00 25431840 /opt/splunk/bin/splunkd MAP: 017a5000-017b9000 rw-p 013a5000 fc:00 25431840 /opt/splunk/bin/splunkd MAP: 017b9000-017f4000 rw-p 00000000 00:00 0 [heap] MAP: 7f05155ea000-7f05155ff000 r-xp 00000000 fc:00 9437447 /lib/x86_64-linux-gnu/libgcc_s.so.1 MAP: 7f05155ff000-7f05157fe000 ---p 00015000 fc:00 9437447 /lib/x86_64-linux-gnu/libgcc_s.so.1 MAP: 7f05157fe000-7f05157ff000 r--p 00014000 fc:00 9437447 /lib/x86_64-linux-gnu/libgcc_s.so.1 MAP: 7f05157ff000-7f0515800000 rw-p 00015000 fc:00 9437447 /lib/x86_64-linux-gnu/libgcc_s.so.1 MAP: 7f0515800000-7f0516c00000 rw-p 00000000 00:00 0 MAP: 7f0516dfd000-7f0516dfe000 ---p 00000000 00:00 0 MAP: 7f0516dfe000-7f0516ffe000 rw-p 00000000 00:00 0 [stack:22393] MAP: 7f0516ffe000-7f0516fff000 ---p 00000000 00:00 0 MAP: 7f0516fff000-7f05171ff000 rw-p 00000000 00:00 0 MAP: 7f05171ff000-7f0517200000 ---p 00000000 00:00 0 MAP: 7f0517200000-7f0517c00000 rw-p 00000000 00:00 0 MAP: 7f0517dfe000-7f0517dff000 ---p 00000000 00:00 0 MAP: 7f0517dff000-7f0517fff000 rw-p 00000000 00:00 0 MAP: 7f0517fff000-7f0518000000 ---p 00000000 00:00 0 MAP: 7f0518000000-7f0519000000 rw-p 00000000 00:00 0 [stack:20998] MAP: 7f05191ff000-7f0519200000 ---p 00000000 00:00 0 MAP: 7f0519200000-7f0519800000 rw-p 00000000 00:00 0 [stack:20972] MAP: 7f05199fd000-7f05199fe000 ---p 00000000 00:00 0 MAP: 7f05199fe000-7f0519bfe000 rw-p 00000000 00:00 0 [stack:20971] MAP: 7f0519bfe000-7f0519bff000 ---p 00000000 00:00 0 MAP: 7f0519bff000-7f0519dff000 rw-p 00000000 00:00 0 [stack:20970] MAP: 7f0519dff000-7f0519e00000 ---p 00000000 00:00 0 MAP: 7f0519e00000-7f051a000000 rw-p 00000000 00:00 0 [stack:20968] MAP: 7f051a000000-7f051a400000 rw-p 00000000 00:00 0 MAP: 7f051a400000-7f051ac00000 rw-p 00000000 00:00 0 MAP: 7f051ad5c000-7f051ad5d000 ---p 00000000 00:00 0 MAP: 7f051ad5d000-7f051ad66000 rw-p 00000000 00:00 0 MAP: 7f051ad66000-7f051ad67000 ---p 00000000 00:00 0 MAP: 7f051ad67000-7f051ad6b000 rw-p 00000000 00:00 0 MAP: 7f051ad6b000-7f051af20000 r-xp 00000000 fc:00 9440973 /lib/x86_64-linux-gnu/libc-2.15.so MAP: 7f051af20000-7f051b11f000 ---p 001b5000 fc:00 9440973 /lib/x86_64-linux-gnu/libc-2.15.so MAP: 7f051b11f000-7f051b123000 r--p 001b4000 fc:00 9440973 /lib/x86_64-linux-gnu/libc-2.15.so MAP: 7f051b123000-7f051b125000 rw-p 001b8000 fc:00 9440973 /lib/x86_64-linux-gnu/libc-2.15.so MAP: 7f051b125000-7f051b12b000 rw-p 00000000 00:00 0 MAP: 7f051b12b000-7f051b143000 r-xp 00000000 fc:00 9440956 /lib/x86_64-linux-gnu/libpthread-2.15.so MAP: 7f051b143000-7f051b342000 ---p 00018000 fc:00 9440956 /lib/x86_64-linux-gnu/libpthread-2.15.so MAP: 7f051b342000-7f051b343000 r--p 00017000 fc:00 9440956 /lib/x86_64-linux-gnu/libpthread-2.15.so MAP: 7f051b343000-7f051b344000 rw-p 00018000 fc:00 9440956 /lib/x86_64-linux-gnu/libpthread-2.15.so MAP: 7f051b344000-7f051b348000 rw-p 00000000 00:00 0 MAP: 7f051b348000-7f051b443000 r-xp 00000000 fc:00 9440969 /lib/x86_64-linux-gnu/libm-2.15.so MAP: 7f051b443000-7f051b642000 ---p 000fb000 fc:00 9440969 /lib/x86_64-linux-gnu/libm-2.15.so MAP: 7f051b642000-7f051b643000 r--p 000fa000 fc:00 9440969 /lib/x86_64-linux-gnu/libm-2.15.so MAP: 7f051b643000-7f051b644000 rw-p 000fb000 fc:00 9440969 /lib/x86_64-linux-gnu/libm-2.15.so MAP: 7f051b644000-7f051b65b000 r-xp 00000000 fc:00 25429406 /opt/splunk/lib/libz.so.1.2.8 MAP: 7f051b65b000-7f051b65c000 rw-p 00016000 fc:00 25429406 /opt/splunk/lib/libz.so.1.2.8 MAP: 7f051b65c000-7f051b6e8000 r-xp 00000000 fc:00 25431661 /opt/splunk/lib/libsqlite3.so.0.8.6 MAP: 7f051b6e8000-7f051b6eb000 rw-p 0008b000 fc:00 25431661 /opt/splunk/lib/libsqlite3.so.0.8.6 MAP: 7f051b6eb000-7f051b6ed000 r-xp 00000000 fc:00 9440971 /lib/x86_64-linux-gnu/libdl-2.15.so MAP: 7f051b6ed000-7f051b8ed000 ---p 00002000 fc:00 9440971 /lib/x86_64-linux-gnu/libdl-2.15.so MAP: 7f051b8ed000-7f051b8ee000 r--p 00002000 fc:00 9440971 /lib/x86_64-linux-gnu/libdl-2.15.so MAP: 7f051b8ee000-7f051b8ef000 rw-p 00003000 fc:00 9440971 /lib/x86_64-linux-gnu/libdl-2.15.so MAP: 7f051b8ef000-7f051bb18000 r-xp 00000000 fc:00 25429404 /opt/splunk/lib/libcrypto.so.1.0.0 MAP: 7f051bb18000-7f051bb42000 rw-p 00228000 fc:00 25429404 /opt/splunk/lib/libcrypto.so.1.0.0 MAP: 7f051bb42000-7f051bb46000 rw-p 00000000 00:00 0 MAP: 7f051bb46000-7f051bcf7000 r-xp 00000000 fc:00 25431658 /opt/splunk/lib/libxml2.so.2.9.1 MAP: 7f051bcf7000-7f051bd01000 rw-p 001b1000 fc:00 25431658 /opt/splunk/lib/libxml2.so.2.9.1 MAP: 7f051bd01000-7f051bd02000 rw-p 00000000 00:00 0 MAP: 7f051bd02000-7f051bd09000 r-xp 00000000 fc:00 9440960 /lib/x86_64-linux-gnu/librt-2.15.so MAP: 7f051bd09000-7f051bf08000 ---p 00007000 fc:00 9440960 /lib/x86_64-linux-gnu/librt-2.15.so MAP: 7f051bf08000-7f051bf09000 r--p 00006000 fc:00 9440960 /lib/x86_64-linux-gnu/librt-2.15.so MAP: 7f051bf09000-7f051bf0a000 rw-p 00007000 fc:00 9440960 /lib/x86_64-linux-gnu/librt-2.15.so MAP: 7f051bf0a000-7f051bf2c000 r-xp 00000000 fc:00 9440961 /lib/x86_64-linux-gnu/ld-2.15.so MAP: 7f051bf36000-7f051bf37000 rw-p 00000000 00:00 0 MAP: 7f051bf37000-7f051bf46000 r-xp 00000000 fc:00 25429410 /opt/splunk/lib/libbz2.so.1.0.3 MAP: 7f051bf46000-7f051bf48000 rw-p 0000e000 fc:00 25429410 /opt/splunk/lib/libbz2.so.1.0.3 MAP: 7f051bf48000-7f051bfd7000 r-xp 00000000 fc:00 25429401 /opt/splunk/lib/libarchive.so.13.1.2 MAP: 7f051bfd7000-7f051bfdb000 rw-p 0008e000 fc:00 25429401 /opt/splunk/lib/libarchive.so.13.1.2 MAP: 7f051bfdb000-7f051bfdd000 rw-p 00000000 00:00 0 MAP: 7f051bfdd000-7f051c03e000 r-xp 00000000 fc:00 25431660 /opt/splunk/lib/libssl.so.1.0.0 MAP: 7f051c03e000-7f051c048000 rw-p 00061000 fc:00 25431660 /opt/splunk/lib/libssl.so.1.0.0 MAP: 7f051c048000-7f051c049000 rw-p 00000000 00:00 0 MAP: 7f051c049000-7f051c097000 r-xp 00000000 fc:00 25431659 /opt/splunk/lib/libxslt.so.1.1.28 MAP: 7f051c097000-7f051c099000 rw-p 0004d000 fc:00 25431659 /opt/splunk/lib/libxslt.so.1.1.28 MAP: 7f051c099000-7f051c0d9000 r-xp 00000000 fc:00 25431644 /opt/splunk/lib/libpcre.so.1.2.1 MAP: 7f051c0d9000-7f051c0da000 rw-p 0003f000 fc:00 25431644 /opt/splunk/lib/libpcre.so.1.2.1 MAP: 7f051c0da000-7f051c0db000 rw-p 00000000 00:00 0 MAP: 7f051c0e2000-7f051c0e3000 rw-p 00000000 00:00 0 MAP: 7f051c0e3000-7f051c116000 r-xp 00000000 fc:00 25431801 /opt/splunk/lib/libjemalloc.so.1 MAP: 7f051c116000-7f051c118000 rw-p 00033000 fc:00 25431801 /opt/splunk/lib/libjemalloc.so.1 MAP: 7f051c118000-7f051c12c000 rw-p 00000000 00:00 0 MAP: 7f051c12c000-7f051c12d000 r--p 00022000 fc:00 9440961 /lib/x86_64-linux-gnu/ld-2.15.so MAP: 7f051c12d000-7f051c12f000 rw-p 00023000 fc:00 9440961 /lib/x86_64-linux-gnu/ld-2.15.so MAP: 7fff53226000-7fff53247000 rw-p 00000000 00:00 0 [stack] MAP: 7fff5325f000-7fff53260000 r-xp 00000000 00:00 0 [vdso] MAP: ffffffffff600000-ffffffffff601000 r-xp 00000000 00:00 0 [vsyscall] Last errno: 0 Threads running: 7 argv: [splunkd -p 8089 start] Process renamed: [splunkd pid=29117] splunkd -p 8089 start [process-runner] Process renamed: [splunkd pid=29117] search --id=1396993215.369 --maxbuckets=0 --ttl=600 --maxout=50000000 --maxtime=8640000 --lookups=1 --reduce_freq=10 --user=mdegan --pro --roles=admin:can_delete:power:user Thread: "dispatch", did_join=1, ready_to_run=Y, main_thread=N First 8 bytes of Thread token @0x7f05185a0010: 00000000 00 f7 3f 18 05 7f 00 00 |..?.....| 00000008 x86 CPUID registers: 0: 0000000B 756E6547 6C65746E 49656E69 1: 00020651 03080800 82982203 1FEBFBFF 2: 55035A01 00F0B2FF 00000000 00CA0000 3: 00000000 00000000 00000000 00000000 4: 00000000 00000000 00000000 00000000 5: 00001000 00001000 00000003 00001120 6: 00000007 00000001 00000009 00000000 7: 00000000 00000000 00000000 00000000 8: 00000000 00000000 00000000 00000000 9: 00000001 00000000 00000000 00000000 A: 07300403 0000007F 00000000 00000000 B: 00000000 00000000 1B139CCD 00000003 80000000: 80000008 00000000 00000000 00000000 80000001: 00000000 00000000 00000001 28100800 80000002: 20202020 49202020 6C65746E 20295228 80000003: 6E6F6558 20295228 20555043 202D3745 80000004: 30373832 20402020 30342E32 007A4847 80000005: 00000000 00000000 00000000 00000000 80000006: 00000000 00000000 01006040 00000000 80000007: 00000000 00000000 00000000 00000100 80000008: 00003028 00000000 00000000 00000000 terminating...

mdegann · ‎01-16-2014

you can use the sendemail command from Splunk which will give you an error level if your syntax is messed up. http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Sendemail.

mdegann · ‎01-14-2014

Ha. Finally an a question I have some experience with, there is a function in sendemail.py def generateHTMLResults(results):. Couple of things to note, len(Reults) = how many events made it into the alert. You can loop through results because its a list of results. Each result is a list of cols, and each column has a value. This should help you start formatting your HTML email.

mdegann · ‎01-08-2014

This usually means that there is no metadata for your Vewstate ID. The easiest way to fix this is to go into your savedsearches.conf and remove the *vsid = ** Lines and do a https://127.0.0.1:8000/en-GB/debug/refresh. This will rebuild all the required VSID's, you could also take the alternate root and write them into your app/metadata/local.meta

Posts	4
Solutions	0
Karma Given	6
Karma Received	8
Member Since	‎08-15-2013

Online Status	Offline
Date Last Visited	‎10-18-2022 06:54 PM

Splunk thread segmentation Fault

Splunk thread segmentation Fault

Re: Modify e-mail alert output

Re: Modify e-mail alert output

Re: "Viewstate object not found" error while cloni...