Here's my setup: 1 search head, 4 indexers, 1 universal forwarder
The UF is trying to index a large file (2G), I'm seeing the "Current data throughput (256kb/s) has reached maxKBps. AS a result data forwarding may be throttled."
If I increase the maxKBps in limits.conf, and restart the UF, will I lose the data that was waiting to be forwarded? Will the Splunk UF pick up in the same spot of the file to keep trying?
My inputs.conf is
[batch:///opt2/ingest/json-data]
index=foo
sourcetype=json-foo
move_policy=sinkhole
and my outputs.conf is:
[tcpout]
defaultGroup=primary_indexers
useACK=true
[tcpout:primary_indexers]
autoLBFrequency=30
server=server1:9997,server2:9997,server3:9997,server4:9997
(I actually have the server ips in the server list above).
Splunk can resume from the position it was on the monitored file.
however with the sinkhole, it should delete the file once compelted.
Do you see the file deleted yet ?
to be safe, I would recommend to make a copy of the file
When you stop splunk, do not kill the process let it empty it's output queue to ensure that no events are missed.
Then if needed, reindex the file later.
Splunk can resume from the position it was on the monitored file.
however with the sinkhole, it should delete the file once compelted.
Do you see the file deleted yet ?
to be safe, I would recommend to make a copy of the file
When you stop splunk, do not kill the process let it empty it's output queue to ensure that no events are missed.
Then if needed, reindex the file later.