After upgrading to Splunk 5.0.1 from 4.3.1, TCP streams of forwarded events began to include lines such as this:
ForwarderInfo build=143156 version=5.0.1 os=Linux arch=x86_64 hostname=splunksrv1 guid=E2DDF88B-2F84-465E-99DE-CA4AA5CD6E71 fwdType=full ssl=false lastIndexer=None
ForwarderInfo usually appears on the first line of a TCP stream, but occasionally appears later in the stream or not at all.
These ForwarderInfo lines confuse our third-party application that receives forwarded events via TCP. The ForwarderInfo lines have a different format than other events and contain not date/time data.
Our outputs.conf remains unchanged since Splunk 4.3.1. We still specify sendCookedData = false. Here is an excerpt from outputs.conf:
[tcpout]
indexAndForward = true
defaultGroup = nothing
[tcpout:nothing]
disabled = false
server = falsefoo.bar.com:9998
dropEventsOnQueueFull = 1
[tcpout:app-event-dest]
server = appsrvr1:6001,appsrvr2:6001
autoLBFrequency = 45
sendCookedData = false
How can we suppress the ForwarderInfo lines in Splunk 5.0.1?
The ForwarderInfo lines are the heartbeats being sent. If you don't need them, you can disable the heartbeats by adding a heartbeatFrequency attribute and setting it to 0. See below:
[tcpout]
indexAndForward = true
defaultGroup = nothing
heartbeatFrequency = 0
I downvoted this post because heartbeat messages are still being sent even with sendcookeddata = false and heartbeatfrequency = 0. this is on version 6.2.7. it'd be really nice if splunk fixed this.
We had a similar problem as joelshprentz, and the above suggestion didn't end up working. Here's a similar thread regarding this same issue:
We had to use the syslog forwarding method to not also get the heartbeat messages (disclaimer: I was doing the forwarding from a Heavy Forwarder, so the syslog forward method works perfectly in that scenario):
outputs.conf
[syslog:target_group]
server = host:port
type = tcp
props.conf
[spec]
TRANSFORMS-syslog=unique_name
transforms.conf
[unique_name]
REGEX = .
DEST_KEY = _SYSLOG_ROUTING
FORMAT = target_group
The above is what worked for us. Good luck.