Getting Data In

Why are my Windows system event logs not getting indexed and getting "WARN TcpOutputProc - Raw connection to ip=serverip:9997 timed out"?

SridharS
Path Finder

Hi, I need to index some Windows system event logs of a remote server (using forwarder) into Splunk. My files are as below in the path $home/etc/apps/<myappname>/local on the forwarder

inputs.conf

[winEventLog://system]
disable = 0
whitelist = 41,104
index = wineventlog

outputs.conf

[tcpout]
default = primary_indexers
[tcpout:primary_indexers]
useACK=true
server = splunkservername:9997

I restarted the forwarder. My tcp port 9997 is open, but why are the logs not getting indexed in Splunk. The last log in the splunkd was:

12-15-2015 14:45:37.097 -0500 WARN  TcpOutputProc - Raw connection to ip=serverip:9997 timed out
12-15-2015 14:45:37.097 -0500 INFO  TcpOutputProc - Ping connection to idx=serverip:9997 timed out. continuing connections
12-15-2015 14:45:37.097 -0500 INFO  TcpOutputProc - Ping connection to idx=serverip:9997 timed out. continuing connections
0 Karma
1 Solution

hortonew
Builder

Has it ever been able to send logs to your indexers? If not, verify that your universal fowarder can:

telnet splunkservername 9997

If that is successful, rule out slow/problematic dns resolutions by configuring your hosts file to map that server name to its IP. Report back after ruling these out. If you try both of these and there are still issues, chances are the problem is on the indexer and not your client sending data. You can also try setting connection_host=false in your inputs.conf for the 9997 stanza, making sure the indexer isn't trying to resolve the name of the forwarder.

View solution in original post

hortonew
Builder

Has it ever been able to send logs to your indexers? If not, verify that your universal fowarder can:

telnet splunkservername 9997

If that is successful, rule out slow/problematic dns resolutions by configuring your hosts file to map that server name to its IP. Report back after ruling these out. If you try both of these and there are still issues, chances are the problem is on the indexer and not your client sending data. You can also try setting connection_host=false in your inputs.conf for the 9997 stanza, making sure the indexer isn't trying to resolve the name of the forwarder.

SridharS
Path Finder

it worked when i added connection_host=false to my inputs.conf

0 Karma

goelt2000
Explorer

hi,

Whats the difference between connection_host=false and connection_host=ip.

I am facing similar errors as mentioned on
https://answers.splunk.com/answers/49833/splunk-forwarder-connection-refused-from-splunk-indexer.htm...

Thanks

0 Karma

hortonew
Builder

From the docs:
connection_host = [ip|dns|none]
* "ip" sets the host to the IP address of the system sending the data.
* "dns" sets the host to the reverse DNS entry for IP address of the system
sending the data.
* "none" leaves the host as specified in inputs.conf, typically the splunk
system hostname.
* Defaults to "dns".

0 Karma

goelt2000
Explorer

none means it will leave it to be the receiver hostname?
What does connection_host=false, means?

Thanks much!

0 Karma

hortonew
Builder

None would be whatever is configured in the forwarder's inputs.conf. So if you have a universal forwarder sending data, whatever is in that universal forwarder's inputs.conf for hostname.

I'm 90% sure none and false do the same thing, but I couldn't find confirmation.

0 Karma
Get Updates on the Splunk Community!

Join Us for Splunk University and Get Your Bootcamp Game On!

If you know, you know! Splunk University is the vibe this summer so register today for bootcamps galore ...

.conf24 | Learning Tracks for Security, Observability, Platform, and Developers!

.conf24 is taking place at The Venetian in Las Vegas from June 11 - 14. Continue reading to learn about the ...

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...