I configured a splunk instance on a linux server and added forwarder to another remote splunk instance. I also configured the receiver on the receiving end.
But I see the following error messages on the forwarder instance.
06-28-2010 19:31:51.226 INFO TcpOutputProc - Retrying connection to xxxx:9997... 06-28-2010 19:31:51.227 WARN TcpOutputProc - Failed to make a connection, will retry.
I restarted the splunk on both forwarding and receiving ends. Is there anything I'm missing?
Sounds like your Splunk indexer does not have Receiving turned on, or there is a problem with network connectivity from your forwarder to the indexer.
First, verify Splunk receiving is enabled on the indexer. Two commands can verify this:
splunk cmd btool inputs list | grep 9997
A listing for you splunk tcp input stanza should be returned.
netstat -an | grep 9997
A "Listen" line should be shown with port 9997.
Second, verify the Splunk forwarder is not blocked from contacting the Splunk indexer:
telnet <splunk_indexer_host> 9997
If you do not get a connection established, then there is likely a deeper problem which will require help from support.
i want to check the network connectivity
but the command grep does'nt work
without knowing your exact configuration it is a bit hard to find out what can be wrong: Is the receiving port the same as the port set up for the forwarder (9997)? Can you ping / telnet to the receiver on port 9997?