All Apps and Add-ons

Splunk Add-on for AWS is not working, s3 generic input not indexing while other s3 generic inputs are working?

Log_wrangler
Builder

I have a number of generic s3 inputs configured and indexing - normally without issue.
I can see in the logs for the working inputs show indexing s3 data is completing.

When I look at the newly created input, I see the same log messages, EXCEPT - indexed s3 data.

message="Start processing."
message="Start processing" last_modified="2019-03-01T00:00:00.000Z" latest_scanned="2019-04-02T21:05:31.000Z"
message="Start of discovering S3 keys."
message="begin loading credentials"
message="load credentials succeed"
message="Create new S3 connection."
message="End of fetching S3 objects."
message="Sweep ckpt file after completion of key discovering."
message="End of processing!"
message="The last data ingestion iteration hasn't been completed yet."

but there is NO message="Indexed S3 files." Like I see with the successful aws inputs. ... and there is no s3 data for that input coming in.

aws add-on is 4.4.0 on Splunk 6.4.1 HF

Can anyone point me in the right direction?

Please advise.

0 Karma

abhiravi7
Engager

Did you figure out the issue behind this? I am stuck with the same issue.

0 Karma

chans28
Explorer

For us it turned out to be the the AWS TA has 4 cacert files that need updating if your companies network has their own SSL certs. 3 of 4 are named conventionally as cacert.pem. However, 1 is named cacert.txt in this
directory

/opt/splunk/etc/apps/Splunk_TA_aws/bin/3rdparty/boto/cacerts/cacerts.txt

Once we updated that with our company's certs everything started working.

0 Karma

suhasinihulikal
Explorer

Did you figure out the issue behind this? I am stuck with the same issue.

chans28
Explorer

Did you ever figure out why this was happening? I have the exact same issue.

kagamalai
Explorer

I am also facing exact same issue 

| message="The last data ingestion iteration hasn't been completed yet."

0 Karma

xiyangyang
Path Finder

I am also facing exact same issue 

Is there any progress about this issue?

0 Karma

kagamalai
Explorer

There is no improvement, but every time changing the configuration and restarted the Splunk service it will take an hour to resume the logs flow 

0 Karma
Get Updates on the Splunk Community!

Splunk APM: New Product Features + Community Office Hours Recap!

Howdy Splunk Community! Over the past few months, we’ve had a lot going on in the world of Splunk Application ...

Index This | Forward, I’m heavy; backward, I’m not. What am I?

April 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

A Guide To Cloud Migration Success

As enterprises’ rapid expansion to the cloud continues, IT leaders are continuously looking for ways to focus ...