Getting Data In

Problem reading syslog events

mmather67
Path Finder

My firewall is using syslog-ng to send logs to my log server over TCP on port 514. In Splunk>>Manager>>Data inputs>>TCP I have one entry, for port 514, which says source=tcp:514x and host=Firewall.

If I set Sourcetype=syslog, one particular log appears with host=2011 instead of host=Firewall.

If instead I set Sourcetype=syslog-ng, most of the time a few events get combined into one.

What should I do?

Tags (1)
0 Karma

mmather67
Path Finder

Excellent. Thanks for your help.

With the proviso that I don't know how to trigger host=2011, so I will wait for one of those events to happen naturally and see what happens.

...local\props.conf now says:

[syslog-ng]

TIME_FORMAT = %Y:%m:%d-%H:%M:%S

SHOULD_LINEMERGE=false

Is there anything else that should be done when changing the sourcetype from syslog to syslog-ng?

I presume, by the way, that the TCP 514 entry in Data Inputs applies before props.conf. Otherwise [syslog-ng] would not be recognised.

0 Karma

mmather67
Path Finder

I cannot pretend to read that. But why is it doing it anyway? What is it hoping to achieve?

0 Karma

Ayn
Legend

The reason you're getting host=2011 when using the "syslog" sourcetype is because Splunk has transforms for that particular sourcetype that sets the host based on log events. Here's the transform that does the job:

[syslog-host]
DEST_KEY = MetaData:Host
REGEX = :\d\d\s+(?:\d+\s+|(?:user|daemon|local.?)\.\w+\s+)*\[?(\w[\w\.\-]{2,})\]?\s
FORMAT = host::$1
0 Karma

JSapienza
Contributor

You might try adding the fllowing stanza to %SPLUNK_HOME\etc\system\local\props.conf

[syslog-ng]
SHOULD_LINEMERGE = False

Bounce splunk and check your events.

0 Karma

mmather67
Path Finder

In response to JSapienza

Syslog only provides single-line events. All examples below are single lines.

inputs.conf has nothing relevant.

When the sourcetype is syslog, this event is picked up properly:-

<190>2011:10:19-16:45:13 reverseproxy: srcip="211.142.x.x" localip="66.207.x.x" size="0" user="-" host="211.142.x.x" method="HEAD" statuscode="200" time="8772" url="/" server="66.207.x.x" referer="-" cookie="-" set-cookie="-"

and this one gets host=2011:-

<190>2011:10:19-16:45:13 reverseproxy: [Wed Oct 19 16:45:13 2011] [warn] [client 211.142.x.x] proxy: no HTTP 0.9 request (with no host line) on incoming request and preserve host set forcing hostname to be 66.207.x.x for uri /

When the sourcetype is syslog-ng, the following two events get picked up as one:-

<30>2011:10:20-06:49:13 ulogd[4729]: id="2002" severity="info" sys="SecureNet" sub="packetfilter" name="Packet accepted" action="accept" fwrule="17" initf="eth1" outitf="eth2" srcmac="0:1e:79:1a:x.x" dstmac="0:1a:8c:11:x.x" srcip="69.165.x.x" dstip="192.168.x.x" proto="6" length="60" tos="0x00" prec="0x00" ttl="56" srcport="60634" dstport="8000" tcpflags="SYN"

<30>2011:10:20-06:49:14 ulogd[4729]: id="2002" severity="info" sys="SecureNet" sub="packetfilter" name="Packet accepted" action="accept" fwrule="10" initf="eth0" outitf="eth2" srcmac="0:21:9b:8e:x.x" dstmac="0:1a:8c:11:x.x" srcip="192.168.x.x" dstip="192.168.x.x" proto="6" length="48" tos="0x00" prec="0x00" ttl="127" srcport="63563" dstport="9997" tcpflags="SYN"

By the way, the local props.conf says:

[source::tcp:514]

TIME_FORMAT = %Y:%m:%d-%H:%M:S

host=Firewall-props

but I don't believe that is relevant.

0 Karma

JSapienza
Contributor

You might have a line format or line breaking issue. Are these multi-line events ? Paste in a few lines from the raw sylog so we can take a look.
What does the Stanza look like in your inputs.conf ? Check %SPLUNK_HOME%\etc\system\local\inputs.conf .

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...