I am seeing errors like below
04-19-2019 12:21:42.676 -0400 ERROR CacheManager - action=download, cacheId="ra|aca_os~427~32226E90-F8B0-4E01-9A5C-54CB63AD5BDC|224A3123-B91A-42BD-B2F6-5F61C172FE67_test_NSc38aff232c7b15e9", status=failed, elapsed_ms=735
04-19-2019 12:21:41.945 -0400 INFO CacheManager - Cache miss for cache_id=ra|aca_os~427~32226E90-F8B0-4E01-9A5C-54CB63AD5BDC|224A3123-B91A-42BD-B2F6-5F61C172FE67_test_NSc38aff232c7b15e9 oldest_evictable_bucket=1555690901.941656
How can I check if these report acceleration searches registered with a remote store?
1) Here is example to see the content of a bucket.
/bin/splunk cmd splunkd rfs -- ls bucket:_internal~163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9
for full paths run: splunkd rfs -- ls --starts-with volume:my_s3_vol/_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/
size,name
5,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/.rawSize
5,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/.sizeManifest4.1
6455,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/1556560541-1556560535-499366360098950950.tsidx
94,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/Hosts.data
180,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/SourceTypes.data
237,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/Sources.data
320,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/Strings.data
284,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/bloomfilter
75,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/bucket_info.csv
945,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/rawdata/journal.gz
8,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/rawdata/slicemin.dat
23,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/rawdata/slicesv2.dat
1614,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/receipt.json
OR
You could also use the command like below provide you know the path to remote location
bin/splunk cmd splunkd rfs -- ls --starts-with volume:my_s3_vol/_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/
size,name
5,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/.rawSize
5,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/.sizeManifest4.1
6455,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/1556560541-1556560535-499366360098950950.tsidx
94,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/Hosts.data
180,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/SourceTypes.data
237,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/Sources.data
320,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/Strings.data
284,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/bloomfilter
75,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/bucket_info.csv
945,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/rawdata/journal.gz
8,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/rawdata/slicemin.dat
23,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/guidSplunk-F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/rawdata/slicesv2.dat
1614,_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/receipt.json
2)Command to fetch the content of bucket from the remote store
/bin/splunk cmd splunkd -- getF bucket:_audit~70~D9F2C312-7136-415A-9E5B-579258D7F626/receipt.json <audit~70~D9F2C312-7136-415A-9E5B-579258D7F626>
Where <audit~70~D9F2C312-7136-415A-9E5B-579258D7F626> is the output directory
OR you may also use
/bin/splunk cmd splunkd rfs -- getF volume:my_s3_vol/_internal/db/94/9b/163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9/receipt.json <_internal~163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9>
Where <_internal~163~F5D45572-D0DF-4DDA-B1BD-A435ADCCBFA9> is the directory
3)To list the content from the summary bucket
…/bin/splunk cmd splunkd rfs -- ls --starts-with volume:my_s3_vol/_audit/dma/e6/ec/79~D9F2C312-7136-415A-9E5B-579258D7F626/6D84278A-CF58-426B-9592-4BB68A7EC5CF_DM_search_internal_audit_logs/
size,name
3139,_audit/dma/e6/ec/79~D9F2C312-7136-415A-9E5B-579258D7F626/6D84278A-CF58-426B-9592-4BB68A7EC5CF_DM_search_internal_audit_logs/guidSplunk-D9F2C312-7136-415A-9E5B-579258D7F626/1556560623-1556560545-5666012404967947423.tsidx
4714,_audit/dma/e6/ec/79~D9F2C312-7136-415A-9E5B-579258D7F626/6D84278A-CF58-426B-9592-4BB68A7EC5CF_DM_search_internal_audit_logs/guidSplunk-D9F2C312-7136-415A-9E5B-579258D7F626/metadata.csv
75,_audit/dma/e6/ec/79~D9F2C312-7136-415A-9E5B-579258D7F626/6D84278A-CF58-426B-9592-4BB68A7EC5CF_DM_search_internal_audit_logs/guidSplunk-D9F2C312-7136-415A-9E5B-579258D7F626/metadata_checksum
831,_audit/dma/e6/ec/79~D9F2C312-7136-415A-9E5B-579258D7F626/6D84278A-CF58-426B-9592-4BB68A7EC5CF_DM_search_internal_audit_logs/receipt.json
To get a file for summary bucket from remote location the below is cli example.
4)Here is the command to retrieve the summary bucket.
/bin/splunk cmd splunkd rfs -- getF volume:my_s3_vol/_audit/dma/e6/ec/79~D9F2C312-7136-415A-9E5B-579258D7F626/6D84278A-CF58-426B-9592-4BB68A7EC5CF_DM_search_internal_audit_logs/receipt.json testrbal
cat ././testrbal/receipt.json
{"objects":[{"name":"./guidSplunk-D9F2C312-7136-415A-9E5B-579258D7F626/1556560623-1556560545-5666012404967947423.tsidx","size":3139},{"name":"./guidSplunk-D9F2C312-7136-415A-9E5B-579258D7F626/metadata.csv","size":4714},{"name":"./guidSplunk-D9F2C312-7136-415A-9E5B-579258D7F626/metadata_checksum","size":75},{"name":"./guidSplunk-D9F2C312-7136-415A-9E5B-579258D7F626/done","size":0}],"manifest":{"summary_path":"/home/rbal/742p2/var/lib/splunk/audit/datamodel_summary/79_D9F2C312-7136-415A-9E5B-579258D7F626/6D84278A-CF58-426B-9592-4BB68A7EC5CF/DM_search_internal_audit_logs","summary_earliest_time":"1556560545","summary_latest_time":"1556560623","summary_size_on_disk":"16384"},"user_data":{"uploader_guid":"D9F2C312-7136-415A-9E5B-579258D7F626","content_hash":"D4AF95215FF87FD3632AF3B04027504B24A30829CFA545689B0C583C39D5B54A"}}[root@centos65-64sup02 db]#