How to add an AWS account to the Splunk Add-on for Amazon Web Services via the Splunk_TA_aws directory?

[cwyse]

We use clustered search heads and clustered forwarders. All the documents on how to set up the AWS account seem to be GUI based. So, we set everything up on one search head. Then copied our Splunk_TA_aws directory to the deployment server and deployed it to all of our forwarders. As that is what the directions for the app say to do. Now we get the following messages:

10-16-2015 22:25:54.800 +0000 ERROR ExecProcessor - message from "python /opt/splunk/etc/apps/Splunk_TA_aws/bin/aws_config.py" ERRORNo AWS Account is configured.

I have checked and we have a passwords.conf with the correct info. This all worked fine when it was on the search heads, but putting things on our heavy forwarders just doesn't seem to work. Anyone know where I can add the AWS account info in the Splunk_TA_aws directory?

[rpille_splunk]

It sounds like you prefer to handle the configuration directly in the add-on. However, if you want to use the configuration experience offered to you in the app, you can do everything in your search heads if you follow the detailed instructions here: http://docs.splunk.com/Documentation/AWS/4.0.0/Installation/Installon-prem#Install_on_a_distributed_.... It's another option, but may not be your preference.

If you are choosing to do your configuration through the add-on rather than through the app:

1. Splunk recommends using Splunk Web to configure your AWS account credentials so that the encryption is handled seamlessly for you. You can observe the results in your local/app.conf. The passwords.conf handles proxy credentials.

2. You should configure your credentials on your data collection nodes, usually one or more heavy forwarders. You do not need the credentials saved on your search heads, because you only need them for data collection. Do not configure data collection on a search head cluster, or your inputs will be duplicated across all members of your cluster resulting in duplicate data collection.

3. Don't use a deployment server to deploy a configured add-on across multiple forwarders. The deployment server is not compatible with credential management or with deploying configured modular inputs to multiple nodes (which results in duplicate data collection.)

The documentation here specifies that we only support AWS account configuration via the add-on through Splunk Web: http://docs.splunk.com/Documentation/AddOns/released/AWS/Setuptheadd-on

You may configure inputs either through Splunk Web or the configuration files. Each input is documented here: http://docs.splunk.com/Documentation/AddOns/latest/AWS/ConfigureInputs

And, like I said before, you can also opt to do all of the configuration through the app instead. That would be entirely through Splunk Web. Those docs are here:
Installation: http://docs.splunk.com/Documentation/AWS/4.0.0/Installation/Installon-prem
Account setup: http://docs.splunk.com/Documentation/AWS/4.0.0/Installation/Setuptheapp
Inputs: http://docs.splunk.com/Documentation/AWS/4.0.0/Installation/Inputsoverview

[cwyse]

To be honest, we would love to use the app. However, the directions for the remote target command allow you to use 1 search head to 1 forwarder. We have 4 search heads and 4 forwarders over an ELB. So while I can set that up. It only means that I would have to go through all 4 search heads and attach each of them to a different forwarder. Then run the same command on all 4. Unless there is something I'm missing?

[mlconnor]

In my AWS account, we have a master account with 90 sub-accounts. We build the Splunk user into the account using automation so it would be nice to generate the passwords.conf file. This is a huge pain for me right now.

If you can tell me the hash algorithm you used to hash the password (e.g. MD5) I could easily generate it.

[rpille_splunk]

Correct, the remote target command only allows you to specify one forwarder target. If that doesn't work for you, then configuring your accounts and inputs directly on your forwarders is the way to go. You can use the app's configuration UI on each heavy forwarder, or you can use the TA's UI or conf files directly on each heavy forwarder. Just be sure to NOT to touch the Configure tab in the app on the search heads at all, in that case.