Hi,
I have some customers who want to take their logfiles and export them, so that they can then be imported into another tool. The files are pretty large, and the exports are taking a while (as is the download). Is there another way to export the files? A way to pipe them (in raw format) to another directory?
Then in that case it has to be incremental searches.
If your only problem is one of export capacity and this is an ongoing requirement, perhaps you could use a scheduled search to export in time-stamped incremental chunks over specified time ranges?
Agree it has to be incremental searches
The customer doesn't have access to the logs, hence the need for Splunk.
From the source (host) itself why dont you send logs to 3rd Party tool as well your Splunk forwarder.
The tool is 3rd party tool that the developers use to do some analysis. We only want -_raw. It's very app specific. Currently, they run the search, and then export the file, which can be very large. I've seen it crash the splunk gui once already.
You might want to give a bit more detail. When you say "export"... what are you doing now? What is this other tool? Does this other tool make use of anything except _raw?