Getting Data In

Automatic Sourcetype Tagging on Syslog Server

R_B
Path Finder

Hi everyone,

In my environment, we are collecting logs from several types of devices on a syslog server, then forwarding the data to the indexers. Most of the data comes into the syslog server on port 514. We have brocade, juniper, f5, cisco, bluecoat, and some others. This example I found to use syslog-ng with Splunk I think is fantastic: https://www.splunk.com/blog/2016/03/11/using-syslog-ng-with-splunk.html

Essentially, the configuration takes the incoming data and automatically separates the data by device type. So the syslog server would put juniper into /logs/juniper, brocade into /logs/brocade, f5 into /logs/f5, etc. Then I could just create an input stanza in Splunk that says anything /juniper tag with the juniper sourcetype, /brocade gets brocade sourcetype, etc.

This is exactly what we would want. My question though, which comes from a lack of understanding syslog in depth, is how would we take all of this different data coming into syslog on port 514 and have syslog automatically determine what type of device the data is coming from? In the example, it looks like that might be what the filters are doing... but if that's true, how would I figure out a filter to identify each device type's data?

Oh and I should add that each device type is using the same standard syslog format I believe, and when looking at the different data, there didn't seem to be any key indicators that identifies what type of device the data is coming from.

R_B
Path Finder

Thank you all for replying with suggestions, my team really appreciates the input! We discussed all of the options. We are very much leaning towards @woodcock's suggestion to use a different port for each sourcetype, and for the devices that can only send on port 514, using a naming convention so that we can specificially apply the sourcetype to those devices. We felt that this would be the most simple, straightforward, and viable option in our environment.

Once we finalize a decision and configure it in our environment, I'll report back here with how it goes, so that it can help anyone else that reads this or runs into a similar situation.

Thanks!

0 Karma

woodcock
Esteemed Legend

Be sure to click Accept and UpVote where appropriate when you come back.

0 Karma

FrankVl
Ultra Champion

Or run multiple instances of rsyslog/syslog-ng on separate (virtual) IPs and segregate the traffic that way, if you’re stuck with using port 514.

woodcock
Esteemed Legend

BRILLIANT! I don't know why I never thought of that!!!!

0 Karma

woodcock
Esteemed Legend

The best way to build a future-proof syslog server is to put each sourcetype on a different port. Most syslog export functions on most applications do allow this. There are a very few big name products that cannot. As much as possible, use this approach and fight back hard against the security ninnies who will complain; there is nothing at all insecure about using more ports. When you cannot, make sure that the hostnames of the servers encode their function; for example, your DCs use a hostnaming convention like location-DC-instance (e.g. DAL-DC-01). Then you can build a hostname filter ( *-DC-* ) to split it out.

thambisetty
SplunkTrust
SplunkTrust

Hi @R_B,

may be your syslog-ng filters are not working properly.

syslog-ng requires to configure source ,filter and destination.

example filter from the document is below
filter f_cisco_asa { match(“%ASA” value(“PROGRAM”)) or match(“%ASA” value(“MESSAGE”)); };

This looks for %ASA in all UDP 514 messages. if it matches then will write to destination
destination d_cisco_asa { file(“/home/syslog/logs/cisco/asa/$HOST/$YEAR-$MONTH-$DAY-cisco-asa.log” create_dirs(yes)); };
using below line.
log { source(s_network); filter(f_cisco_asa); destination(d_cisco_asa); };

————————————
If this helps, give a like below.

pradeepkumarg
Influencer

Can you not use source field to differentiate the data by device?

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...