Are you saving the frozen buckets to a local directory first with coldtofrozendir? I pushed out an app with just this script to my indexers, and im not seeing anything in my s3 bucket. When i run the script manually, I get bin>./smartstore2frozen.sh
./smartstore2frozen.sh: line 5: $1: unbound variable When I comment out 'set -u' I get bin>./smartstore2frozen.sh
date: invalid date ‘@’ Heres the script. The only thing i modified is that I removed the proxy portion and changed the bucket name #!/bin/bash
set -e
set -u
bucket=$1
instance=$(hostname -s)
region=us-east-1
s3bucket=<my s3 bucket>
NOW=$(date +"%Y-%m-%d")
LOG=/opt/splunk/var/log/splunk/smartstore2frozen-${NOW}.log
#Gets index name and warm bucket name from path passed by splunk
index=$(echo $1 | cut -f7 -d"/")
warm=$(echo $1 | cut -f9 -d"/")
#Converts the epoch time from the warm bucket name to EST
startEpoch=$(echo $warm | cut -f3 -d"_")
endEpoch=$(echo $warm | cut -f2 -d"_")
startDate=$(date -d @$startEpoch '+%m_%d_%Y')
endDate=$(date -d @$endEpoch '+%m_%d_%Y')
#Sets AWS Sginature Version - Needed for S3 KMS-SSE
aws configure set s3.signature_version s3v4
#Creates log file
touch ${LOG}
echo "bucket to move: " $bucket >> $LOG
#Copies bucket to S3 and logs the output along with timestamps
/usr/bin/aws s3 cp ${bucket} s3://${s3bucket}/frozen/${index}/${startDate}_to_${endDate}/${warm} --recursive --region ${region} 2>&1 | tr "\r" "\n" > >(awk '{print strftime("%Y-%m-%d:%H:%M:%S ") $0}' >> $LOG)
... View more