My firewall is using syslog-ng to send logs to my log server over TCP on port 514. In Splunk>>Manager>>Data inputs>>TCP I have one entry, for port 514, which says source=tcp:514x and host=Firewall.
If I set Sourcetype=syslog, one particular log appears with host=2011 instead of host=Firewall.
If instead I set Sourcetype=syslog-ng, most of the time a few events get combined into one.
What should I do?
Excellent. Thanks for your help.
With the proviso that I don't know how to trigger host=2011, so I will wait for one of those events to happen naturally and see what happens.
...local\props.conf now says:
[syslog-ng]
TIME_FORMAT = %Y:%m:%d-%H:%M:%S
SHOULD_LINEMERGE=false
Is there anything else that should be done when changing the sourcetype from syslog to syslog-ng?
I presume, by the way, that the TCP 514 entry in Data Inputs applies before props.conf. Otherwise [syslog-ng] would not be recognised.
I cannot pretend to read that. But why is it doing it anyway? What is it hoping to achieve?
The reason you're getting host=2011 when using the "syslog" sourcetype is because Splunk has transforms for that particular sourcetype that sets the host based on log events. Here's the transform that does the job:
[syslog-host]
DEST_KEY = MetaData:Host
REGEX = :\d\d\s+(?:\d+\s+|(?:user|daemon|local.?)\.\w+\s+)*\[?(\w[\w\.\-]{2,})\]?\s
FORMAT = host::$1
You might try adding the fllowing stanza to %SPLUNK_HOME\etc\system\local\props.conf
[syslog-ng]
SHOULD_LINEMERGE = False
Bounce splunk and check your events.
In response to JSapienza
Syslog only provides single-line events. All examples below are single lines.
inputs.conf has nothing relevant.
When the sourcetype is syslog, this event is picked up properly:-
<190>2011:10:19-16:45:13 reverseproxy: srcip="211.142.x.x" localip="66.207.x.x" size="0" user="-" host="211.142.x.x" method="HEAD" statuscode="200" time="8772" url="/" server="66.207.x.x" referer="-" cookie="-" set-cookie="-"
and this one gets host=2011:-
<190>2011:10:19-16:45:13 reverseproxy: [Wed Oct 19 16:45:13 2011] [warn] [client 211.142.x.x] proxy: no HTTP 0.9 request (with no host line) on incoming request and preserve host set forcing hostname to be 66.207.x.x for uri /
When the sourcetype is syslog-ng, the following two events get picked up as one:-
<30>2011:10:20-06:49:13 ulogd[4729]: id="2002" severity="info" sys="SecureNet" sub="packetfilter" name="Packet accepted" action="accept" fwrule="17" initf="eth1" outitf="eth2" srcmac="0:1e:79:1a:x.x" dstmac="0:1a:8c:11:x.x" srcip="69.165.x.x" dstip="192.168.x.x" proto="6" length="60" tos="0x00" prec="0x00" ttl="56" srcport="60634" dstport="8000" tcpflags="SYN"
<30>2011:10:20-06:49:14 ulogd[4729]: id="2002" severity="info" sys="SecureNet" sub="packetfilter" name="Packet accepted" action="accept" fwrule="10" initf="eth0" outitf="eth2" srcmac="0:21:9b:8e:x.x" dstmac="0:1a:8c:11:x.x" srcip="192.168.x.x" dstip="192.168.x.x" proto="6" length="48" tos="0x00" prec="0x00" ttl="127" srcport="63563" dstport="9997" tcpflags="SYN"
By the way, the local props.conf says:
[source::tcp:514]
TIME_FORMAT = %Y:%m:%d-%H:%M:S
host=Firewall-props
but I don't believe that is relevant.
You might have a line format or line breaking issue. Are these multi-line events ? Paste in a few lines from the raw sylog so we can take a look.
What does the Stanza look like in your inputs.conf ? Check %SPLUNK_HOME%\etc\system\local\inputs.conf .