I have just turned on compression and have over 100 GB of uncompressed data. How can I compress it and Splunk still be able to read it?
Let me rephrase. I have 100GB of already indexed uncompressed data. I have just configured compression from the forwarders to the indexers. How do I compress this data so splunk can read it in order to save some disk space?
Neverminf, I misunderstood your question.
indexes already compressed when indexed. I just compressed the data for transfer.
Just point Splunk at the file. Splunk reads compressed files, but has to go through a single threaded process to decompress each compress file first and then read it in. Why not let Splunk read in uncompressed data and then compress it? Otherwise it would take a long time to read all that compressed data. It's not out of the ordinary to do it though. We see this all the time.