After upgrading to 5.01 we began receiving this error.
received event for unconfigured/disabled/deleted index='_audit' with source='source::audittrail' host='host::foo' sourcetype='sourcetype::audittrail' (1 missing total)
Looking at the Indexes I can see the _audit index is disabled with the current size of the file being 0MB.
I tried setting _audit to Enable but receive and error message that: One or more indexes could not be initialized and were automatically disabled, please see splunkd.log for more details
Looking the spunkd.log file this is what is reported:
11-30-2012 13:09:27.747 -0800 INFO IndexProcessor - reloading index config: request received
11-30-2012 13:09:27.749 -0800 INFO IndexProcessor - reloading index config: start
11-30-2012 13:09:27.749 -0800 INFO IndexProcessor - request state change from=RUN to=RECONFIGURING
11-30-2012 13:09:27.749 -0800 INFO IndexProcessor - Initializing: readonly=false reloading=true
11-30-2012 13:09:27.754 -0800 INFO IndexProcessor - Got a list of count=1 added, modified, or removed indexes
11-30-2012 13:09:27.755 -0800 INFO IndexProcessor - Reloading index config: shutdown subordinate threads, now restarting
11-30-2012 13:09:27.755 -0800 INFO IndexProcessor - indexes.conf - indexThreads param autotuned to=2
11-30-2012 13:09:27.755 -0800 INFO HotDBManager - idx=_audit Setting hot mgr params: maxHotSpanSecs=7776000 snapBucketTimespans=false maxHotBuckets=3 maxDataSizeBytes=786432000 quarantinePastSecs=77760000 quarantineFutureSecs=2592000
11-30-2012 13:09:27.755 -0800 INFO databasePartitionPolicy - idx=_audit Initialized with params='[300,60,188697600,,,,786432000,5,true,500000,5,5,false,3,0,_blocksignature,7776000,1000000,0,3,77760000,2592000,131072,25,0,15,0,0,-1,18446744073709551615,2592000,true,60000,300000,false]' isSlave=false needApplyDeleteJournal=false
11-30-2012 13:09:27.756 -0800 ERROR DatabaseDirectoryManager - idx=_audit bucket=hot_v1_0 Detected directory manually copied into its database, causing id conflicts [path1='C:\Program Files\Splunk\var\lib\splunk\audit\db\db_1326238803_1326231564_0' path2='C:\Program Files\Splunk\var\lib\splunk\audit\db\hot_v1_0'].
11-30-2012 13:09:27.756 -0800 ERROR DatabaseDirectoryManager - idx=_audit bucket=hot_v1_20 Detected directory manually copied into its database, causing id conflicts [path1='C:\Program Files\Splunk\var\lib\splunk\audit\db\db_1331855014_1331854207_20' path2='C:\Program Files\Splunk\var\lib\splunk\audit\db\hot_v1_20'].
11-30-2012 13:09:27.756 -0800 ERROR IndexProcessor - caught exception for idx=_audit during initialization: 'idx=_audit bucket=hot_v1_20 Detected directory manually copied into its database, causing id conflicts [path1='C:\Program Files\Splunk\var\lib\splunk\audit\db\db_1331855014_1331854207_20' path2='C:\Program Files\Splunk\var\lib\splunk\audit\db\hot_v1_20'].'.Disabling the index, please fix-up and run splunk enable index
11-30-2012 13:09:27.759 -0800 ERROR IndexProcessor - One or more indexes could not be initialized and were automatically disabled, please see splunkd.log for more details
11-30-2012 13:09:27.764 -0800 INFO IndexProcessor - request state change from=RECONFIGURING to=RUN
11-30-2012 13:09:27.764 -0800 INFO IndexProcessor - reloading index config: end
Any help to correct this would be appreciated.
Thank you
Doug
... View more