Reporting

best way to export logfiles

a212830
Champion

Hi,

I have some customers who want to take their logfiles and export them, so that they can then be imported into another tool. The files are pretty large, and the exports are taking a while (as is the download). Is there another way to export the files? A way to pipe them (in raw format) to another directory?

Tags (1)
0 Karma

strive
Influencer

Then in that case it has to be incremental searches.

0 Karma

grijhwani
Motivator

If your only problem is one of export capacity and this is an ongoing requirement, perhaps you could use a scheduled search to export in time-stamped incremental chunks over specified time ranges?

0 Karma

strive
Influencer

Agree it has to be incremental searches

0 Karma

a212830
Champion

The customer doesn't have access to the logs, hence the need for Splunk.

0 Karma

strive
Influencer

From the source (host) itself why dont you send logs to 3rd Party tool as well your Splunk forwarder.

0 Karma

a212830
Champion

The tool is 3rd party tool that the developers use to do some analysis. We only want -_raw. It's very app specific. Currently, they run the search, and then export the file, which can be very large. I've seen it crash the splunk gui once already.

0 Karma

rsennett_splunk
Splunk Employee
Splunk Employee

You might want to give a bit more detail. When you say "export"... what are you doing now? What is this other tool? Does this other tool make use of anything except _raw?

With Splunk... the answer is always "YES!". It just might require more regex than you're prepared for!
0 Karma
Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Wondering How to Build Resiliency in the Cloud?

IT leaders are choosing Splunk Cloud as an ideal cloud transformation platform to drive business resilience,  ...

Updated Data Management and AWS GDI Inventory in Splunk Observability

We’re making some changes to Data Management and Infrastructure Inventory for AWS. The Data Management page, ...