Is it possible to use AWS tags for scheduled searches?

[lnx11]

Hi
We are using Splunk App for AWS with Splunk 6.5.
We want to be able to monitor Linux log files for certain keywords and hostgroups.
Idea is to group hosts per their AWS owner tags.
I was wondering if we can directly use AWS tag values in scheduled searches so we can properly create and forward alerts per the hostgroups they were generated for?
Thank you

[kaufmanm · Accepted solution]

Yes, it is possible. You will need to join data from your log with the tagging data output from source=ec2_instances. So if the hostname was both in the host field of your log and in a tag of the EC2 instance labeled hostname, you could do something like:

source="/var/log/messages" ERROR | eval tags.hostname = host | join type=left tags.hostname [search sourcetype="aws:description" source=*ec2_instances* earliest=-60m | fields tags.hostname tags.owner]

Then you will have your ERROR message with the tags.owner field you need to escalate the alert. You could pass the event to a script that knows how to process that field appropriately or do a lookup to find the e-mail address. Left join since when the join fails you probably still want to proceed with a default escalation.

[lnx11]

Hi, Thank you for your answer.
I just ran a test search.
I can't seem to access sourcetype="aws:description". Only sourcetype available in search results is of "syslog".

[kaufmanm]

The aws:description sourcetype is from a configured input in the Splunk TA for AWS. Either you are not collecting the data/the input is not configured or you are not searching against the correct index. Try adding index=* to the subsearch, e.g. index=* sourcetype="aws:description". Otherwise look into configuring the description input here: http://docs.splunk.com/Documentation/AddOns/released/AWS/DescriptionInput

[lnx11]

I did not have any inputs in AWS add-on, so I added one via web ui (I was greeted via: "Configuring this add-on on a search head is not best practice." warning), following the instructions in the link you provided.
Most everything was pre-selected, I picked aws region, iam-role etc, left the "index" value at "default".
When I execute search, I get the results without aws tags.owner.

I am able to execute below search from splunk search app and get the info including tags etc.
index="" sourcetype="aws:description" source=":ec2_instances" earliest=-5m

I guess, issue I have is joining the two searches from two different sources?
Thank you

[kaufmanm]

hostname and owner were example tags from my environment. In the source=ec2_instances data you will have to see what tags you have available to you or set some more in AWS in order to make a join happen.

[lnx11]

Appreciate your help, thank you!