How to confirm a udp input is running?

[a212830]

Hi,

I'm having issues with what should be a very basic setup. I have an appliance sending syslog messages to a heavy forwarder, on port 514, using UDP. I've verified that the events are coming in via tcpdump. My inputs is setup to listen on port 514, and nothing else is listening on it, but the events are not appearing in the indexer. I've checked for all-time, and recent time, and manually send some events via netcat. I do not see anything in the logs indicating that splunk is even listening for this data. Should some message appear somewhere, indicating that it's listening on port 514, similar to how it shows what logs are being watched? The HFW can talk to the indexer, as internal events are appearing.

Inputs:

[udp://514]
connection_host = dns
index = main
sourcetype=syslog 
disabled = no
queueSize = 1KB

[mattymo · Accepted solution]

is this HF on Centos or RHEL?? firewalld/iptables all good? also 514 is privileged, is Splunk root? might have to dance around that a bit

firewall-cmd --list-all

firewall-cmd --permanent --zone=public --add-port=514/udp

systemctl restart firewalld.service

[sloshburch]

ahem: Worst Practices...and How to Fix Them
Start at 3min in.

You know I'm always willing to give a good hassle to ya! 😉

[mattymo]

no doubt! @a212830, now that we got ya up and running, you will want to explore items like, not running splunk as root, using syslog receivers like rsyslog or syslog-ng to put data to disk and pick it up with a UF or check out options for scale using HEC!

http://conf.splunk.com/sessions/2017-sessions.html#search=HEC%20with%20syslog&

[a212830]

Indeed! Actually, this isn't mine... a friend in another group was trying to get his data into a different BU's Splunk, and they weren't able to get it done, so he tried it, and I finished it for him. He's been advised not to run as root...

[a212830]

running as root. RH7.

[mattymo]

did you create a rule in firewalld for udp 514?

[a212830]

It's udp, and I see the events coming in via tcpdump.

[mattymo]

oops. updated. tcpdump is a good start but sees the packers before they are dropped.

[a212830]

But, yes, you are right. Firewalld is the culprit. Apparently it's enabled by default on RH7, but no RH6.

Thanks!

[mattymo]

nice! will update the answer with the command, was about to post it