compression from a forwarder to an indexer

[mfrost8]

We are looking at deploying some Splunk lightweight forwarders to servers that are remote. As such, we're interested in reducing the network bandwidth required for the LWF's to transmit to the indexers. Our understanding is that the forwaders in question have more CPU available to them than they'd have bandwidth so compression potentially makes sense.

Currently we do no compression between forwarders and indexers.

I see the 'compressed' option in outputs.conf is how you'd turn this on on the forwarder side. However, according to the docs, it looks like you have to turn this on on the indexer's listener port as well.

I have a few questions.

1) I assume that this means that I can't somehow have my existing listener perform double-duty -- handling both compressed and uncompressed data. So I'd have to setup a second listener that handles only compressed traffic from forwarders.

2) I'm a little confused where this indexer listener gets configured. I don't see an option to turn on compression in the web interface. I grep'd around and it seems our existing listener settings are in etc/apps/search/local/inputs.conf. It looks like I'd configure the secondary listener with compression in this file (assuming I'm correct about needing a secondary listener for compressed traffic).

Thanks

[fwilmot]

http://answers.splunk.com/questions/6513/compressed-data-from-forwarder-to-indexer lists what your inputs.conf and outputs.conf might look like so you will know where to enable compression, and what the command syntax is.

[gkanapathy · Accepted solution]

1. You can just set up a second Splunk input port number on the indexer for compressed data. A forwarder would send to either the compressed listen port or the uncompressed one appropriately, but both would be on the same indexer.

2. You have to configure this in the outputs.conf file on the forwarder and the inputs.conf file on the indexer. It is not in the GUI.