I recently setup Kinesis Firehose to push to Splunk HEC which is ingesting fine, however, I would like to have the logs sent to "nonprod" or "prod" index depending on the cloudwatch log group name.
For example, if cloudwatch log group name is like "prod", go to prod-index. Else, go to nonprod-index.
I'm utilizing this python script for my lambda function (Cloudwatch2FH2HEC.py) and tried editing the area below but I believe this is nested in the lower JSON and will not matter.
Edit: Hyperlink didn't work
https://github.com/ptdavies17/CloudwatchFH2HEC
https://github.com/ptdavies17/CloudwatchFH2HEC/blob/master/Cloudwatch2FH2HEC.py
Working version:
if "CloudTrail" in loggrp:
sourcetype="aws:cloudtrail"
elif "VPC" in loggrp:
sourcetype="aws:cloudwatchlogs:vpcflow"
else:
sourcetype=os.environ['SPLUNK_SOURCETYPE']
return_message = '{"time": ' + str(log_event['timestamp']) + ',"host": "' + logstrm + '","source": "' + loggrp + '"'
return_message = return_message + ',"sourcetype":"' + sourcetype + '"'
return_message = return_message + ',"event": ' + json.dumps(log_event['message']) + '}\n'
return return_message + '\n'
Attempted version:
if "CloudTrail" in loggrp:
sourcetype="aws:cloudtrail"
elif "VPC" in loggrp:
sourcetype="aws:cloudwatchlogs:vpcflow"
else:
sourcetype=os.environ['SPLUNK_SOURCETYPE']
if "prod" in str.lower(loggrp):
index="prod-kinesis"
else:
index="nonprod-kinesis"
return_message = '{"time": ' + str(log_event['timestamp']) + ',"host": "' + logstrm + '","source": "' + loggrp + '"'
return_message = return_message + ',"sourcetype":"' + sourcetype + '"'
return_message = return_message + ',"index":"' + index + '"'
return_message = return_message + ',"event": ' + json.dumps(log_event['message']) + '}\n'
return return_message + '\n'
Any thoughts on how to achieve this?
... View more