Splunk Enterprise

Splunk forwarder shows Cooked connection to ip=xxxxx:9997 timed out

sanjayagrey
New Member

My configuration
1 forwarder
2 indexer
2 search heads
The config files on forwarder are as below
cat inputs.conf
[monitor:////var/logs/myserver.log]
disabled = false
sourcetype = mysourcetye
index=myindex

outputs.conf
[tcpout:xxxx]
server=server1.com:9997,server2:9997
autoLB = true
autoLBFrequency = 300
forceTimebasedAutoLB = true
useACK = true

On indexer, the inputs.conf is in /opt/splunk/etc/apps/myapp/local
cat inputs.conf
[splunktcp://9997]
disabled = 1

The server.conf in /opt/splunk/etc/system/local location has following stanza
[general]
pass4SymmKey = $1$xxxxxxx
serverName = myserver.com

[clustering]
master_uri = https://myclustermaster.com:8089
mode = slave

[license]
master_uri = https://mylicensemaster.com:8089

I am seeing following error in forwarder splunkd.log

07-14-2016 11:58:09.776 +0100 INFO WatchedFile - Will begin reading at offset=966525 for file='/var/xxx/logs/jetty/jetty.log'.
07-14-2016 11:58:09.794 +0100 INFO WatchedFile - Will begin reading at offset=316928 for file='/opt/splunkforwarder/var/log/splunk/metrics.log'.
07-14-2016 11:58:09.968 +0100 INFO WatchedFile - File too small to check seekcrc, probably truncated. Will re-read entire file='/opt/splunkforwarder/var/log/splunk/splunkd_stdout.log'.
07-14-2016 11:58:09.969 +0100 INFO WatchedFile - File too small to check seekcrc, probably truncated. Will re-read entire file='/opt/splunkforwarder/var/log/splunk/mongod.log'.
07-14-2016 11:58:09.971 +0100 INFO WatchedFile - Will begin reading at offset=9129 for file='/opt/splunkforwarder/var/log/splunk/splunkd-utility.log'.
07-14-2016 11:58:09.974 +0100 INFO WatchedFile - File too small to check seekcrc, probably truncated. Will re-read entire file='/opt/splunkforwarder/var/log/splunk/license_usage.log'.
07-14-2016 11:58:09.976 +0100 INFO WatchedFile - Will begin reading at offset=3230 for file='/opt/splunkforwarder/var/log/splunk/conf.log'.
07-14-2016 11:58:09.978 +0100 INFO WatchedFile - Will begin reading at offset=1230 for file='/opt/splunkforwarder/var/log/splunk/splunkd_stderr.log'.
07-14-2016 11:58:10.004 +0100 INFO WatchedFile - File too small to check seekcrc, probably truncated. Will re-read entire file='/opt/splunkforwarder/var/log/splunk/remote_searches.log'.
07-14-2016 11:58:10.006 +0100 INFO WatchedFile - File too small to check seekcrc, probably truncated. Will re-read entire file='/opt/splunkforwarder/var/log/splunk/scheduler.log'.
07-14-2016 11:58:10.010 +0100 INFO WatchedFile - File too small to check seekcrc, probably truncated. Will re-read entire file='/opt/splunkforwarder/var/log/splunk/splunkd_ui_access.log'.
07-14-2016 11:58:10.045 +0100 INFO WatchedFile - File too small to check seekcrc, probably truncated. Will re-read entire file='/opt/splunkforwarder/var/log/splunk/searchhistory.log'.
07-14-2016 11:58:10.048 +0100 INFO WatchedFile - Will begin reading at offset=68593 for file='/opt/splunkforwarder/var/log/splunk/audit.log'.
07-14-2016 11:58:29.697 +0100 WARN TcpOutputProc - Cooked connection to ip=Inderxer1:9997 timed out
07-14-2016 11:58:49.697 +0100 WARN TcpOutputProc - Cooked connection to ip=indexer2:9997 timed out

Tags (1)
0 Karma

sanjayagrey
New Member

Resolved this with
cat inputs.conf
[splunktcp://9997]
disabled = 0

0 Karma

jplumsdaine22
Influencer

Hah I didn't notice the input was disabled the first time. Glad you were able to solve the problem. Do you mind marking this as answered?

0 Karma

jplumsdaine22
Influencer

Just means that the forwarder couldn't make a connection to the indexer specified. Check your network (ie can you make a connection to the indexer with telnet/nc ?)

0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...