All Apps and Add-ons

Can anyone please help me with Splunk App for AWS?

ashishebansal
New Member

I am facing issue as per which S3 dashboard option in Splunk App for AWS is not loading any data. The Splunk documentation says that S3 data should be on-boarded using "sourcetype=aws:s3:accesslogs", however the query in S3 traffic tab has "aws:cloudtrai"l in it. I tried changing it to "aws:s3:accesslogs" but query is still not working out. I tried almost everything but it seems like a complicated issue as per which I will have to rebuilt the entire dashboard.

So, is it something that is a miss out from Splunk or myself?

0 Karma
1 Solution

amiracle
Splunk Employee
Splunk Employee

First, make sure that you are sending your AWS data to the main index, if you created your own 'aws' index, then make sure to update the macros so that the summary indexes are properly populated. Here is a troubleshooting guide to help you.

Once you have set that up, then the queries should work and the source type 'aws:cloudtrail' will show the S3 access since the query is using the cloud trail API calls to S3 to show the access of the events.

That specific dashboard is using data models which are populated with the saved searches and macros mentioned in the troubleshooting guide.

View solution in original post

0 Karma

amiracle
Splunk Employee
Splunk Employee

First, make sure that you are sending your AWS data to the main index, if you created your own 'aws' index, then make sure to update the macros so that the summary indexes are properly populated. Here is a troubleshooting guide to help you.

Once you have set that up, then the queries should work and the source type 'aws:cloudtrail' will show the S3 access since the query is using the cloud trail API calls to S3 to show the access of the events.

That specific dashboard is using data models which are populated with the saved searches and macros mentioned in the troubleshooting guide.

0 Karma

ashishebansal
New Member

I have tried most of the things to make that S3 dashboard work but it isn't.

Just to explain it little more, I am sending the data to main index with sourcetype="aws:s3:accesslogs" but it is still not working out. I am trying to work on some POC that will need this app as well, if possible then can we connect regarding this on call or email?

Query one the view in this dashboard is using.


(index="main" OR (index="main" OR index="default") OR index="default") sourcetype="aws:cloudtrail"
| lookup all_eventName eventName OUTPUTNEW function
| search function="S3 Data Event"
| spath output=bucketName path="requestParameters.bucketName"
| spath output=objectName path=requestParameters.key
| spath output=userName path=userIdentity.userName
| eval error=if(errorCode=="success",0, 1)
| lookup unauthorized_errorCode errorCode OUTPUT Unauthorized
| eval Unauthorized=if(Unauthorized=="true", 1, 0)
| stats count by region, aws_account_id, bucketName, objectName, userName, eventName, userAgent, sourceIPAddress,Unauthorized, error, readOnly,_time
| search (aws_account_id="") (region="")
| stats sum(count) as count sum(error) as errors
| nadefault errors count
| eval _total_name="Data Events", _total_field=count


0 Karma

amiracle
Splunk Employee
Splunk Employee

There seems to be an issue with the field extraction for the userName. Try setting the 6th line to :
| spath output=userName path=userIdentity.sessionContext.sessionIssuer.userName

The search should complete with the count and errors tables populating. Here is the complete search, I took out the last 'search' since I'm not running it within the context of the search filters on the dashboard:

(index="main" OR (index="main" OR index="default") OR index="default") sourcetype="aws:cloudtrail" 
| lookup all_eventName eventName OUTPUTNEW function 
| search function="S3 Data Event"
| spath output=bucketName path="requestParameters.bucketName" 
| spath output=objectName path=requestParameters.key 
| spath output=userName path=userIdentity.sessionContext.sessionIssuer.userName
| eval error=if(errorCode=="success",0, 1) 
| lookup unauthorized_errorCode errorCode OUTPUT Unauthorized
| eval Unauthorized=if(Unauthorized=="true", 1, 0) 
| stats count by region, aws_account_id, bucketName, objectName, userName, userAgent, sourceIPAddress,Unauthorized, error, readOnly, _time
| stats sum(count) as count sum(error) as errors 
| nadefault errors count 
| eval _total_name="Data Events", _total_field=count
0 Karma

ashishebansal
New Member

Hi Amiracle,

Thank you so much for providing that solution.

I followed your approach and was able to fix most of the issues but "Most Recent Modifications - Latest 10" is still not working out.

Below is the query:


(index="main" OR (index="main" OR index="default") OR index="default") sourcetype="aws:cloudtrail"
| lookup all_eventName eventName OUTPUTNEW function
| search function="S3 Data Event"
| spath output=bucketName path="requestParameters.bucketName"
| spath output=objectName path=requestParameters.key
| spath output=userName path=userIdentity.sessionContext.sessionIssuer.userName
| eval error=if(errorCode=="success",0, 1)
| lookup unauthorized_errorCode errorCode OUTPUT Unauthorized
| eval Unauthorized=if(Unauthorized=="true", 1, 0)
| stats count by region, aws_account_id, bucketName, objectName, userName, eventName, userAgent, sourceIPAddress,Unauthorized, error, readOnly,_time
| search (aws_account_id="") (region="")
| search readOnly=false (eventName="DeleteObject" OR eventName="PutObject" OR eventName="PostObject")
| sort - _time limit=10
| eval _time=strftime(_time, "%m/%d/%Y %H:%M:%S")
| table bucketName, objectName, userName, eventName, sourceIPAddress, userAgent, _time

| rename bucketName as BucketName, objectName as ObjectName, userName as UserName, eventName as EventName, sourceIPAddress as "Source IP Address", userAgent as UserAgent, _time as Time

The other tab "S3 - Traffic Analysis" is not working at all.

0 Karma

amiracle
Splunk Employee
Splunk Employee

S3 Traffic Analysis is based on the data model: S3_Access_Log (Settings -> Data Models) and see if the data model was built. If not, hit the rebuild button or update it.

0 Karma

ashishebansal
New Member

Yes, sure thanks a lot for help. Is there anything that you can suggest me to try before the model rebuilt ?

0 Karma

amiracle
Splunk Employee
Splunk Employee

No, all the data is contained in the model so changing any of the searches would just lead to missing fields etc. Are you sending the data into Splunk using the SQS Based S3 input or are you using another method?

0 Karma

ashishebansal
New Member

I am using the SQS based S3 only with below configurations.

[aws_sqs_based_s3://S3_Logs]
aws_account = RoleName
aws_iam_role = RoleName
interval = 300
s3_file_decoder = S3AccessLogs
sourcetype = aws:s3:accesslogs
sqs_batch_size = 10
sqs_queue_region = us-east-1
sqs_queue_url = queueName

disabled = 0

0 Karma

ashishebansal
New Member

Any solution for "Most Recent Modifications - Latest 10" from S3 - Data events that is also not working out.

0 Karma

amiracle
Splunk Employee
Splunk Employee

Yes, check the saved search "CloudTrail S3 Data Event Search" and see if it has completed with results. (Settings -> Search, reports and Alerts -> Find "CloudTrail S3 Data Event Search" and then click on Results.) If it's disabled, enable the search.

0 Karma

ashishebansal
New Member

Yes, it is enabled and has completed the results.

0 Karma

amiracle
Splunk Employee
Splunk Employee

That is the base search that is being used by this dashboard to help populate the different reports.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...