I have further debugged this issue but I am still stumped. Please note I am NOT a JSON programmer.
The issue appearss to be caused by the the SQS message received by Splunk. Here is the line of code that generates the KeyError from aws_cloudtrail.py:
bucket_name = message['s3Bucket']
The key "S3Bucket" does not appear to exist. I dumped the SQS Messsage by adding this line to aws_cloudtrail.py:
logger.log(logging.INFO, "**** message: %s",json.dumps(message))
Here is the output (with some information obfuscated):
2015-01-24 11:06:11,299 INFO pid=9081 tid=MainThread file=aws_cloudtrail.py:process_notifications:295 | **** message: {"Records": [{"requestParameters": {"sourceIPAddress": "0.0.0.0"}, "userIdentity": {"principalId": "AWS:ARxxxxxxxxxxxxxxxxxxxx:i-69999999"}, "eventVersion": "2.0", "s3": {"bucket": {"ownerIdentity": {"principalId": "AAAAAAAAAAAAAA"}, "name": "cloudtrail", "arn": "arn:aws:s3:::cloudtrail"}, "s3SchemaVersion": "1.0", "object": {"eTag": "axxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx", "size": 8450, "key": "Folder1/Folder2/Folder3/us-west-2/2015/01/21/688888888888_CloudTrail_us-west-2_20150121T1000Z_XXXXXXXXXXXXXX.json.gz"}, "configurationId": "Splunk"}, "eventSource": "aws:s3", "responseElements": {"x-amz-id-2": "+XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX=", "x-amz-request-id": "AAAAAAAAAAAAAAAA"}, "eventTime": "2015-01-21T10:00:03.710Z", "awsRegion": "us-west-2", "eventName": "ObjectCreated:Put"}]}
(continued in next comment)
... View more