All Apps and Add-ons

Splunk Add-on for AWS is not working, s3 generic input not indexing while other s3 generic inputs are working?

Log_wrangler
Builder

I have a number of generic s3 inputs configured and indexing - normally without issue.
I can see in the logs for the working inputs show indexing s3 data is completing.

When I look at the newly created input, I see the same log messages, EXCEPT - indexed s3 data.

message="Start processing."
message="Start processing" last_modified="2019-03-01T00:00:00.000Z" latest_scanned="2019-04-02T21:05:31.000Z"
message="Start of discovering S3 keys."
message="begin loading credentials"
message="load credentials succeed"
message="Create new S3 connection."
message="End of fetching S3 objects."
message="Sweep ckpt file after completion of key discovering."
message="End of processing!"
message="The last data ingestion iteration hasn't been completed yet."

but there is NO message="Indexed S3 files." Like I see with the successful aws inputs. ... and there is no s3 data for that input coming in.

aws add-on is 4.4.0 on Splunk 6.4.1 HF

Can anyone point me in the right direction?

Please advise.

0 Karma

abhiravi7
Engager

Did you figure out the issue behind this? I am stuck with the same issue.

0 Karma

chans28
Explorer

For us it turned out to be the the AWS TA has 4 cacert files that need updating if your companies network has their own SSL certs. 3 of 4 are named conventionally as cacert.pem. However, 1 is named cacert.txt in this
directory

/opt/splunk/etc/apps/Splunk_TA_aws/bin/3rdparty/boto/cacerts/cacerts.txt

Once we updated that with our company's certs everything started working.

0 Karma

suhasinihulikal
Explorer

Did you figure out the issue behind this? I am stuck with the same issue.

chans28
Explorer

Did you ever figure out why this was happening? I have the exact same issue.

kagamalai
Explorer

I am also facing exact same issue 

| message="The last data ingestion iteration hasn't been completed yet."

0 Karma

xiyangyang
Path Finder

I am also facing exact same issue 

Is there any progress about this issue?

0 Karma

kagamalai
Explorer

There is no improvement, but every time changing the configuration and restarted the Splunk service it will take an hour to resume the logs flow 

0 Karma
Get Updates on the Splunk Community!

Wondering How to Build Resiliency in the Cloud?

IT leaders are choosing Splunk Cloud as an ideal cloud transformation platform to drive business resilience,  ...

Updated Data Management and AWS GDI Inventory in Splunk Observability

We’re making some changes to Data Management and Infrastructure Inventory for AWS. The Data Management page, ...

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...