Noob - Can't add TCP Port 9997 - Error in handler 'raw'

[franklovecchio]

So, I'm new, and having a bit of trouble 🙂

I have a Splunk instance running, we'll call it my server (can access GUI), that I'm trying to configure to listen on port 9997. I have another box which is setup as a "forwarder", and to configure it, I ran "splunk add forward-server serverIP:9997" and "splunk set splunkd-port 9997" (I changed the mgmt port because not changing it didn't work either).

So, from the GUI on the server, I click "Manage", "Data Inputs", "TCP", and I try to add a new port to receive data on (9997). When I say add syslog from all incoming hosts on this port, I get the error "Encountered the following error while trying to save: In handler 'raw': Parameter name: TCP port 9997 is not available". Why would this be? I'm on amazon ec2, and definitely have the ports 9997, 8000, and 8089 opened. Please help!

[Ayn]

You're mixing different types of inputs here. I'm unsure as to whether that in itself would cause the problems you describe, but when receiving forwarded data from another Splunk instance, you should configure a corresponding receiver rather than a 'raw' data input. Go to Manager -> Forwarding and receiving -> Configure receiving -> Add new. Since you have established connections on port 9997 on the server it seems someone might already have done this!

[netwrkr]

netstat -tnap | grep 9997

anything else currently bound to that port?

[franklovecchio]

On forwarder:
ttcp 0 0 0.0.0.0:9997 0.0.0.0:* LISTEN 14095/splunkd

tcp 0 0 FORWARDERIP:33750 SERVERIP:9997 TIME_WAIT -

tcp 0 0 FORWARDERIP:32878 SERVERIP:9997 ESTABLISHED 14095/splunkd

tcp 0 0 FORWARDERIP:33749 SERVERIP:9997 TIME_WAIT -

tcp 0 0 FORWARDERIP:33751 SERVERIP:9997 ESTABLISHED 14095/splunkd

[franklovecchio]

I don't think so - looks about right to me!

On server:
tcp 0 0 SERVERIP:9997 FORWARDERIP:33749 ESTABLISHED 10923/splunkd

tcp 0 0 SERVERIP:9997 FORWARDERIP:32878 ESTABLISHED 10923/splunkd