Splunk Search

Aws dns field extraction

martinnepolean
Explorer

We are trying to do field extraction of the aws dns events, currently we are getting the events with below indexname, source and sourcetype

index = aws-cloudtrail source = us-east-1:/aws/route53/ins.company.com:Z7ZW0F4AW5AIB/IAH50-C3 sourcetype = aws:cloudwatch

I have created props and transforms as separate app for field extraction but it is not working

cat props.conf - for some reason * is not showing in this editor Its (source::asterisk/aws/route53/asterisk/asterisk)
[source::/aws/route53//*]
REPORT-fields = AWS_DNS_route53

cat transforms.conf
[AWS_DNS_route53]
DELIMS = " "
FIELDS = "version","query_timestamp","hosted_zoneid","queryname","querytype","response_code","protocol","edge_location","ip_address","subnet"

_RAW

1.0 2020-01-07T13:13:00Z Z7ZW0F4AW5AIB ins.company.com AAAA NOERROR UDP KJS50-C3 2001:1890:1ff:8c7:124:10:98:184 -

0 Karma

to4kawa
Ultra Champion
| makeresults 
| eval _raw="version query_timestamp hosted_zoneid queryname querytype response_code protocol edge_location ip_address subnet
1. 2020-01-07T13:13:00Z Z7ZW0F4AW5AIB ins.company.com AAAA NOERROR UDP KJS50-C3 2001:1890:1ff:8c7:124:10:98:184 -"
| multikv forceheader=1

props.conf:

[source::*/aws/route53/*/*]
EXTRACT-aws_dns_route53 = (?<version>[^ ]+) (?<query_timestamp>[^ ]+) (?<hosted_zoneid>[^ ]+) (?<queryname>[^ ]+) (?<querytype>[^ ]+) (?<response_code>[^ ]+) (?<protocol>[^ ]+)

Hi, @martinnepolean
Why not try it in props.conf because it can be extracted neatly?

0 Karma

martinnepolean
Explorer
[source::*/aws/route53/*/*]
EXTRACT-aws_dns_route53 = (?<version>[^ ]+) (?<query_timestamp>[^ ]+) (?<hosted_zoneid>[^ ]+) (?<queryname>[^ ]+) (?<querytype>[^ ]+) (?<response_code>[^ ]+) (?<protocol>[^ ]+) (?<edge_location>[^ ]+) (?<ip_address>[^ ]+) (?<subnet>[^ ]+)

Tried above props.cong but not working

0 Karma

to4kawa
Ultra Champion
[source::us-east-1:/aws/route53/ins.company.com:Z7ZW0F4AW5AIB/IAH50-C3]

Since the behavior of the asterisk is unknown, why not write it directly once?

0 Karma

martinnepolean
Explorer

Only /aws/route53/ is common in the source and others will change. My props and transforms.conf are working in my test env where I manually feed the raw event in a text file and ingest into splunk. I am facing this issue in prod where we are getting data from sqs based s3 using aws addon.

0 Karma

to4kawa
Ultra Champion

I see.
I don't know much. I'm sorry.

0 Karma

martinnepolean
Explorer

Thanks, it is app permission issue and my props and transforms.conf is working

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...