Splunk App for AWS: Will my current configuration for multiple AWS accounts work?

[JasonCarter80]

We have 9 accounts in AWS. I have set up AWS Config on each account, pointing to their own SNS topic. I have one SQS Queue in us-east-1 subscribed to the SNS topic from each region, which should allow Splunk to watch a single endpoint for notifications as setting up all 9 regions for all 9 accounts seems like it would be an excessive drain on my Splunk server to ping these on a regular basis.

A few questions:

• Will this work as configured?
• If it will not work, how bad of a performance hit am I going to have if I setup all 9 accounts against all 9 regions? Should I simply raise the default checking time?

[_d_ · Accepted solution]

Will this work as configured?
Yes, as long as the credentials you configure Splunk with have the right permissions on the SQS queue you should be fine.

[JasonCarter80]

I believe I have found hard limit on the number of AWS Config items allowed to be populated, which is 30, as I have loaded all 9*9, but only see 3 accounts with all 9, and then a 4th account with 3. If i remove some regions, the config shifts but the overall number is still 30.

Is this configurable? Will this also be in place for CloudWatch and CloudTrail as well?

[JasonCarter80]

How does the initial inventory get populated? Currently I am only seeing the EC2 instances from a single region.

[jzhong_splunk]

Hi Jason,

The initial inventory get populated by triggering a AWS Config Snapshot. When you add a Config input, the snapshot will be triggered automatically, unless your IAM user don't have such permission.

Setting up config service for all regions and all 9 accounts really is a lot of work. So, let me share some good news (I am the dev manager of this app at Splunk, BTW): 1) in the upcoming 4.1 AWS app which will be available in 1 or 2 months, we will be using AWS REST API to list all resources, so you don't have to configure the Config, unless you want to use the topology tool. 2)AWS itself is enhancing the AWS config service so that you can use 1 region to capture all inventory in all regions. 3) recently AWS config has published a feature to let you capture the IAM user/group/role/permission. We are in the process of enhancing topology tool to visualize those IAM resources.

For the time being, you may have to create 9*9 SNS topics, 9*9 SQS queues. You may use CloudFormation or similar tool to automate it. Or you can wait for 1 or 2 months for the next release of AWS app, but with 4.0 app you can still monitor other data sources, such as CloudWatch, billing, VPC flow log.

Hope it helps

[JasonCarter80]

That does help a great deal, i will be spending my morning configuring 9*9 inputs for the time being, thank you for such a great app.

[jzhong_splunk]

Glad to hear such feedback. Happy new year, and you will enjoy the next release of this great app.