Getting Data In

using a forwarder as a high-availability buffer?

tedder
Communicator

I have a couple of indexers behind a heavy forwarder, which reads from a batch of ports (and a few directories). If both of the indexers go down, incoming data is dropped on the floor. Is it possible to configure my forwarder to queue messages? It's bored, since it is just doing some basic parsing and throwing data towards the indexers. I can't be the first person to consider doing this.

1 Solution

gkanapathy
Splunk Employee
Splunk Employee

It is possible, but not recommended. You can review settings in outputs.conf to enable persistent queueing of data. Note that it's an especially bad idea if it's reading from files (vs transient network inputs), as the file content as well as the buffer will take up space on the machine. Also note that it's more likely for a single heavy forwarder to go down than for both a pair of indexers.

View solution in original post

gkanapathy
Splunk Employee
Splunk Employee

It is possible, but not recommended. You can review settings in outputs.conf to enable persistent queueing of data. Note that it's an especially bad idea if it's reading from files (vs transient network inputs), as the file content as well as the buffer will take up space on the machine. Also note that it's more likely for a single heavy forwarder to go down than for both a pair of indexers.

gkanapathy
Splunk Employee
Splunk Employee

In most cases you really ought to have the indexes.conf settings configured so that indexers can't run out of space, and should discard the oldest data when it reaches the limits. This can be difficult if you have very limited storage however and lots of indexes.

0 Karma

tedder
Communicator

it's more likely that both indexers will run out of space- DAMHIK! Sort of a stampede issue- one runs out of space, so more traffic to the second causes issues. Thanks for the tips; knowing it is 'not recommended' is actually very helpful.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...