Getting Data In

Documented recipe for replicating subsets of data to 3rd party system throws errors in 4.3.1 and 4.3.2

Wiggy
Splunk Employee
Splunk Employee

In 4.3.1 and 4.3.2, when using the documented procedure to replicate subsets of data to 3rd party systems found here:

http://docs.splunk.com/Documentation/Splunk/latest/Deploy/Routeandfilterdatad#Replicate_a_subset_of_...

and using the following code recipe in $splunk_home/etc/system/local/outputs.conf:

[tcpout]
defaultGroup = nothing

[tcpout:firstlocation]
disabled = false
server = foo.bar1.com:9998
sendCookedData = false
indexAndForward = true

[tcpout:secondlocation]
disabled = false
server = foo.bar2.com:9998
sendCookedData=false
indexAndForward = true

splunk creates a large amount of errors in splunkd.log in this format:

04-24-2012 09:14:44.078 -0500 ERROR splunklogger - Uncaught exception in pipeline execution (tcp-output-generic-processor) - getting next event
04-24-2012 09:14:44.078 -0500 ERROR pipeline - Runtime exception in pipeline: indexerPipe, processor: tcp-output-generic-processor, error: vector::_M_range_check
04-24-2012 09:14:44.078 -0500 ERROR splunklogger - Uncaught exception in pipeline execution (tcp-output-generic-processor) - getting next event
04-24-2012 09:14:44.078 -0500 ERROR pipeline - Runtime exception in pipeline: indexerPipe, processor: tcp-output-generic-processor, error: vector::_M_range_check

and results in the data not being written to any index.

1 Solution

Wiggy
Splunk Employee
Splunk Employee

This is a known bug in 4.3.1 and 4.3.2. The bug is SPL-50576 and has been fixed and will be released with the 4.3.3 version of splunk. In the mean-time, the work around is as follows:

You must define a [tcpout:nothing], give it a false address, and tell it to drop events on queue full as shown in below example:

[tcpout]
defaultGroup = nothing

[tcpout:nothing]
disabled = false
server = falsefoo.bar.com:9998
indexAndForward = true 
dropEventsOnQueueFull = 1

[tcpout:firstlocation]
disabled = false
server = foo.bar1.com:9998
sendCookedData = false
indexAndForward = true

[tcpout:secondlocation]
disabled = false
server = foo.bar2.com:9998
sendCookedData=false
indexAndForward = true

This will circumvent the problem with routing to a non-existant group and index all data as normal, also routing any other data you have set up to go to the other output stanzas as expected.

View solution in original post

Wiggy
Splunk Employee
Splunk Employee

This is a known bug in 4.3.1 and 4.3.2. The bug is SPL-50576 and has been fixed and will be released with the 4.3.3 version of splunk. In the mean-time, the work around is as follows:

You must define a [tcpout:nothing], give it a false address, and tell it to drop events on queue full as shown in below example:

[tcpout]
defaultGroup = nothing

[tcpout:nothing]
disabled = false
server = falsefoo.bar.com:9998
indexAndForward = true 
dropEventsOnQueueFull = 1

[tcpout:firstlocation]
disabled = false
server = foo.bar1.com:9998
sendCookedData = false
indexAndForward = true

[tcpout:secondlocation]
disabled = false
server = foo.bar2.com:9998
sendCookedData=false
indexAndForward = true

This will circumvent the problem with routing to a non-existant group and index all data as normal, also routing any other data you have set up to go to the other output stanzas as expected.

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...