All Apps and Add-ons

Splunk App and Add-on for Amazon Web Services: Why are Cloudwatch log groups missing from VPC flow logs input?

sudheerchamarth
Explorer

I have installed the Splunk App for AWS, but when I try to configure vpc flow logs, it is not listing all the log groups. Instead, it is only showing a few log groups that start with /aws/lambda/XXXX . Why am not able to list all the log groups?

Splunk version = 6.4.1
Splunk Add-on for Amazon Web Services version = 4.0.0
Splunk App for AWS = 4.2.0

Manual configuration tried:

aws_cloudwatch_logs_tasks.conf

[xxx-Cloudwatch Logs]
account = xxx-xxxx
delay = 1800
groups = xxx-log-group-name
index = default
interval = 600
only_after = 1970-01-01T00:00:00
region = us-east-1
sourcetype = aws:cloudwatchlogs:vpcflow
stream_matcher = .*
0 Karma

rarsan_splunk
Splunk Employee
Splunk Employee

Alternatively, if Splunk HTTP Event Collector (HEC) is enabled in your particular Splunk deployment, consider instead streaming CloudWatch Logs into Splunk via Lambda, that is CloudWatch Logs --> Lambda --> Splunk HEC, as explained in this blog post:
http://blogs.splunk.com/2017/02/03/how-to-easily-stream-aws-cloudwatch-logs-to-splunk/

In addition to near real-time data ingestion, you could benefit from automated configuration management. More specifically, instead of having to manually create a CloudWatch Logs input in Splunk for each logs group, you can have these Lambda functions automatically created (part of CloudFormation template, AWS CLI or other methods) and stream to the same Splunk HEC input.

0 Karma

pchen_splunk
Splunk Employee
Splunk Employee

It's a known issue in aws app 4.2.0. It will be part of our 4.2.1 maintenance release. Or you can ask support to fix it by looking at JIRA AWSAPP-968.
Workaround before 4.2.1:
1. Ask support to follow AWSAPP-968 to fix
2. Use addon to configure this input

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...