Monitoring Splunk

During startup recovery, error reading from process runner child: Bad file number. Splunk fsck failed with error code '8'.

zliu
Splunk Employee
Splunk Employee

When startup and recovering from a unclean shutdown.
Perform recovery now? [y/n] y
Recovering (across all data)...
Error reading from process runner child: Bad file number
Splunk fsck failed with error code '8'. Please file a case online at http://www.splunk.com/page/submit_issue

running splunk as a heavy forwarder without indexing on a some AIX systems with a non root user called splkadm.

Splunk 4.2.1

Tags (1)
1 Solution

Rob
Splunk Employee
Splunk Employee

This is an issue specific to AIX causing the fsck utility to fail.

If you do not wish to see this error you may wish to delete the meta.dirty file that is located in the $SPLUNK_HOME/var/lib/splunk/defaultdb/db/ directory.

Keep in mind that this will only delete the file that was generated on an unclean shutdown and does not actually repair the buckets. To do so you may wish to run the fsck utility manually with the following command:

$SPLUNK_HOME/bin/splunk cmd splunkd fsck --all --mode metadata --repair

Otherwise, have a look at the following answer which describes how to validate the metadata files as well as the bucket tsidx files.

http://splunk-base.splunk.com/answers/5374/how-to-quickly-validate-the-metadata-files-of-a-given-ind...

View solution in original post

Rob
Splunk Employee
Splunk Employee

This is an issue specific to AIX causing the fsck utility to fail.

If you do not wish to see this error you may wish to delete the meta.dirty file that is located in the $SPLUNK_HOME/var/lib/splunk/defaultdb/db/ directory.

Keep in mind that this will only delete the file that was generated on an unclean shutdown and does not actually repair the buckets. To do so you may wish to run the fsck utility manually with the following command:

$SPLUNK_HOME/bin/splunk cmd splunkd fsck --all --mode metadata --repair

Otherwise, have a look at the following answer which describes how to validate the metadata files as well as the bucket tsidx files.

http://splunk-base.splunk.com/answers/5374/how-to-quickly-validate-the-metadata-files-of-a-given-ind...

zliu
Splunk Employee
Splunk Employee

It could be caused by Splunk on AIX 6.x.
Splunk is not certified to work on AIX 6.x.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...