We are trying to do field extraction of the aws dns events, currently we are getting the events with below indexname, source and sourcetype
index = aws-cloudtrail source = us-east-1:/aws/route53/ins.company.com:Z7ZW0F4AW5AIB/IAH50-C3 sourcetype = aws:cloudwatch
I have created props and transforms as separate app for field extraction but it is not working
cat props.conf - for some reason * is not showing in this editor Its (source::asterisk/aws/route53/asterisk/asterisk)
[source::/aws/route53//*]
REPORT-fields = AWS_DNS_route53
cat transforms.conf
[AWS_DNS_route53]
DELIMS = " "
FIELDS = "version","query_timestamp","hosted_zoneid","queryname","querytype","response_code","protocol","edge_location","ip_address","subnet"
_RAW
1.0 2020-01-07T13:13:00Z Z7ZW0F4AW5AIB ins.company.com AAAA NOERROR UDP KJS50-C3 2001:1890:1ff:8c7:124:10:98:184 -
| makeresults
| eval _raw="version query_timestamp hosted_zoneid queryname querytype response_code protocol edge_location ip_address subnet
1. 2020-01-07T13:13:00Z Z7ZW0F4AW5AIB ins.company.com AAAA NOERROR UDP KJS50-C3 2001:1890:1ff:8c7:124:10:98:184 -"
| multikv forceheader=1
props.conf:
[source::*/aws/route53/*/*]
EXTRACT-aws_dns_route53 = (?<version>[^ ]+) (?<query_timestamp>[^ ]+) (?<hosted_zoneid>[^ ]+) (?<queryname>[^ ]+) (?<querytype>[^ ]+) (?<response_code>[^ ]+) (?<protocol>[^ ]+)
Hi, @martinnepolean
Why not try it in props.conf
because it can be extracted neatly?
[source::*/aws/route53/*/*]
EXTRACT-aws_dns_route53 = (?<version>[^ ]+) (?<query_timestamp>[^ ]+) (?<hosted_zoneid>[^ ]+) (?<queryname>[^ ]+) (?<querytype>[^ ]+) (?<response_code>[^ ]+) (?<protocol>[^ ]+) (?<edge_location>[^ ]+) (?<ip_address>[^ ]+) (?<subnet>[^ ]+)
Tried above props.cong but not working
[source::us-east-1:/aws/route53/ins.company.com:Z7ZW0F4AW5AIB/IAH50-C3]
Since the behavior of the asterisk is unknown, why not write it directly once?
Only /aws/route53/ is common in the source and others will change. My props and transforms.conf are working in my test env where I manually feed the raw event in a text file and ingest into splunk. I am facing this issue in prod where we are getting data from sqs based s3 using aws addon.
I see.
I don't know much. I'm sorry.
Thanks, it is app permission issue and my props and transforms.conf is working