Deployment Architecture

How to restore frozen data when it looks my buckets have corrupted files?

guimilare
Communicator

Hi all.

I'm running Splunk 6.3.2 in Linux machines.

I have to restore some old data to Splunk.
I've followed the steps described at http://docs.splunk.com/Documentation/Splunk/6.3.2/Indexer/Restorearchiveddata but I'm facing an error that I can't solve.

I've copied the frozen bucket db_1438454243_1438385486_4557 to /var/splunk/defaultdb/thaweddb/
After that, I executed the following command:

/opt/splunk/bin/splunk rebuild /var/splunk/defaultdb/thaweddb/db_1438454243_1438385486_4557 
USAGE: splunk rebuild <bucketPath> [<indexName>] [--ignore-read-error] [--no-log] The <indexName> parameter is ignored if provided. Please see 'splunk fsck' for more options.  This command is just a wrapper for 'splunk fsck'.

Redirecting to 'splunkd fsck' with args:
        repair --one-bucket --include-hots --bucket-path=/var/splunk/defaultdb/thaweddb/db_1438454243_1438385486_4557
--log-to--splunkd-log ERROR JournalSlice - Error reading compressed journal while streaming: bad gzip header, provider=/var/splunk/defaultdb/thaweddb/db_1438454243_1438385486_4557/rawdata/journal.gz ERROR BucketBuilder - Error reading rawdata: Error reading compressed journal while streaming: bad gzip header, provider=/var/splunk/defaultdb/thaweddb/db_1438454243_1438385486_4557/rawdata/journal.gz WARN  Fsck - Repair (entire bucket) idx= bucket='/var/splunk/defaultdb/thaweddb/db_1438454243_1438385486_4557' failed: (entire bucket) Rebuild for bkt='/var/splunk/defaultdb/thaweddb/db_1438454243_1438385486_4557' failed: Error reading rawdata: Error reading compressed journal while streaming: bad gzip header, provider=/var/splunk/defaultdb/thaweddb/db_1438454243_1438385486_4557/rawdata/journal.gz Rebuilding bucket failed

Inside /var/splunk/defaultdb/thaweddb/db_1438454243_1438385486_4557/rawdata there are only 3 files: slicesv2.dat, slicemin.dat and journal.gz.

It looks like the buckets have corrupted files, since the command file in linux five me that the journal.gz file is data.
Is that correct or I'm missing something?

Thank in advance.
Best regards.

0 Karma

Masa
Splunk Employee
Splunk Employee

Yes, sounds like your bucket is corrupted.

How about "gunzip journal.gz" Does this work without error? Most likely this will return error, too.
Unfortunately, if that's the case, there is no tool available recover the bucket.

somesoni2
Revered Legend
0 Karma
Get Updates on the Splunk Community!

Updated Team Landing Page in Splunk Observability

We’re making some changes to the team landing page in Splunk Observability, based on your feedback. The ...

New! Splunk Observability Search Enhancements for Splunk APM Services/Traces and ...

Regardless of where you are in Splunk Observability, you can search for relevant APM targets including service ...

Webinar Recap | Revolutionizing IT Operations: The Transformative Power of AI and ML ...

The Transformative Power of AI and ML in Enhancing Observability   In the realm of IT operations, the ...