Getting Data In

In Splunk, where and when are data.gz files unzipped?

Log_wrangler
Builder

If I have an app on a heavy forwarder that is pulling in .gz files and sending them to my indexers (i.e. distributed environment), when and where are the files unzipped?

Are the files unzipped at the heavy forwarder or sent zipped to the indexers?

I am trying to troubleshoot why I am seeing unreadable garbage in the search head with a source of file.gz...

It appears that the file was not unzipped.

Please advise.

Thank you

0 Karma

MuS
SplunkTrust
SplunkTrust

Hi Log_wrangler,

Compressed files (like .gz and .zip) are handled by the Archive processor, and are processed in serial. The Archive processor reads the compressed file during the input phase, on your HWF, and the HWF will send the uncompressed events to the indexer.

Unreadable garbage could be either you are having actual unreadable garbage in the file or the indexer is listening on TCP <YourPortNumberHere> instead of SplunkTCP. Sometimes it could also be related to forwarder sending using SSL and receiver not using SSL or vice versa.

Hope this helps ...

cheers, MuS

0 Karma

MuS
SplunkTrust
SplunkTrust

Just another hint regarding the garbage events, check the charset of the file and set it in the props.conf. See the docs for more http://docs.splunk.com/Documentation/Splunk/latest/Troubleshooting/Garbledevents#Symptom

cheers, MuS

0 Karma

Log_wrangler
Builder

So this issue is specifically related to an app residing on a HF.
The Splunk_TA_microsoft-cloudservices unzips files in tables inputs but not blobs inputs.
We can unzip the file with the indexer directly (as a local file input) so it appears the problem is in the app.
Any other ideas how to find the root cause?

Thanks

0 Karma

Log_wrangler
Builder

thank you for your response.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...