Getting Data In

Can't add UDP input because of error "UDP port 514 is not available." Why?

the_wolverine
Champion

Got this question today that would be useful to answer here:

I'm trying to set up Splunk to accept syslog data from my Cisco routers. When I do so, by attempting to add a data input for UDP port 514, I receive the following error:

Encountered the following error while trying to save: In handler 'udp': UDP port 514 is not available.

1 Solution

the_wolverine
Champion

The obvious reason for this error is that port 514 is already in use on the Splunk Host. Splunk is unable to listen on a port that is already in use by another application.

Disable whatever application is listening on port 514 (syslog?).

View solution in original post

luisarcila10
New Member

In new firewall-cmd you must do next step as root to redirect port 514 to 5514, so logs will be recibed at port 5514:

firewall-cmd --zone=public --add-masquerade --permanent
firewall-cmd --zone=public --add-forward-port=port=514:proto=udp:toport=5514 --permanent

Then add in web interface > setup > input data > udp > add input data
create the listening port "5514" next and set type of logs "syslog" next next... finish

0 Karma

fibber12345
New Member

Title

For the bright spark that suggested this file (which is what I am looking for), which one?
find |grep inputs.conf|grep -v READ
./system/local/inputs.conf
./system/default/inputs.conf
./apps/search/local/inputs.conf
./apps/SplunkUniversalForwarder/default/inputs.conf

@Gabe wrote: "sudo ./splunk start from the /Applications/splunk/bin folder."
Now you have really stitched the end-user up.

Default Splunk installation is in /opt/splunkforwarder/ on Linux, and likely most O/S other than Windows and Macs.

0 Karma

tpaulsen
Contributor

The default Splunk installation path did change with v4.x, depending on what kind of Forwarder you use - heavy, lightweight, or universal.

0 Karma

jlunk
Engager

Run (on linux) netstat -nltp and netstat -nlup. This should show you what is listening on each TCP and UDP port, respectively. It is very likely that syslog is in fact already running on port 514. Disabling syslog completely is probably not what you would want to do to remedy this, because it takes care of the local logging service. Without syslog you would not have /var/log/messages. To just turn off syslog's network bindings (port 514), then refer to your OS's syslog documentation.

I_AM_MIKE
Engager

On a Mac (which is BSD based) try the following:

netstat -lvnp udp | grep -e 'pid' -e '514'

ps -ax | grep --pid number here--

0 Karma

gabedimeglio
Engager

If your using a Mac, then simply sudo root when you run Splunk. This bypasses the limitation mentioned by tpaulson of a non root user not having the ability to listen to ports below 1024. So assuming your path is the default, the command would be as follows.

sudo ./splunk start

from the /Applications/splunk/bin folder.

🙂 Gabe

snevarezh
Explorer

this worked for me

0 Karma

aortiz1
New Member

i tried that way and it did not work. do you think i need to open my por 514 from my router? i used netstat -nltp and i did not see any app using that port. it did not show the port 514 there.

Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:8089 0.0.0.0:* LISTEN 19293/splunkd

tcp 0 0 0.0.0.0:443 0.0.0.0:* LISTEN 2060/apache2

tcp 0 0 127.0.0.1:3306 0.0.0.0:* LISTEN 2799/mysqld

tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 2060/apache2

tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 25867/sshd

0 Karma

tpaulsen
Contributor

If you have Splunk installed with a non-root user on LINUX, you can´t listen to ports below 1024. But....there is a work around:

/usr/sbin/iptables -t nat -A PREROUTING -m udp -p udp --dport 514 -j REDIRECT --to-ports 5447

This command uses the IPTABLES Firewall to route UDP from port 514 to port 5447. It needs to be executed as root. And your sysadmin should make it bootable, so that this redirect is permanent and gets loaded even after a system reboot.

Your inputs.conf should look like this then:

[udp://5447]
sourcetype = networking-syslog
queue = parsingQueue

MillerTime
Splunk Employee
Splunk Employee

That command will work as a one-shot, but I don't think it will persist through a restart. Here's an example of what you can put into your IPTables file for the NAT to persist:

Redirect syslog to splunk

-A PREROUTING -p udp -m udp --dport 514 -j REDIRECT --to-ports 5514
COMMIT

Allow Syslog

-A RH-Firewall-1-INPUT -p udp -m udp --dport 5514 -j ACCEPT

0 Karma

gkanapathy
Splunk Employee
Splunk Employee

Another common reason is that you are on a unixy system, and you're not running Splunk as root. These systems require a process to be root to be bind to a port below 1024. There are some other ways around this, but they are all very OS-specific.

nick405060
Motivator

I think this answer is the answer most people are looking for, over the accepted answer. Thanks!

0 Karma

BunnyHop
Contributor

There are scanners that can detect the open port of a host. Try to run a scanner, i.e. nmap, nessus, etc, to see what's using the port 514 on your host (or where you have your splunk installed).

the_wolverine
Champion

The obvious reason for this error is that port 514 is already in use on the Splunk Host. Splunk is unable to listen on a port that is already in use by another application.

Disable whatever application is listening on port 514 (syslog?).

mrenner
Engager

Splunk HF won't let you receive data on UDP 514 even if nothing else is using the port. I just verified this by disabling syslog and verifying in netstat -nlup. I also restarted Splunk and it still wont let you receive on udp 514

0 Karma

scelikok
SplunkTrust
SplunkTrust

If you are running Splunk as non-root user, it cannot start listening ports below 1024. 

If this reply helps you an upvote and "Accept as Solution" is appreciated.

cain1288
Explorer

Yes, this was the issue I had as well running Splunk on 8.2.1

Splunk was unable to ingest syslog data while the application was running as the user Splunk. Running as root has resolved the issue.

A more appropriate technical resolution would be identifying if we could set the user Splunk to be able to listen on ports lower than 1024

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...