Getting Data In

Netscreen Firewall Syslog Input not line breaking

Dragonnet
New Member

I know this problem has already been addressed but I cannot resolve the problem using the directions in 'Juniper Netscreen TCP Syslog messages not breaking properly'

I have added the entries in the two conf files as listed there

You also might need to set a line breaker defined in your sourcetype as

follows $SPLUNK_HOME/etc/system/local/inputs.conf

*[tcp://9999]
sourcetype = juniper_syslog_stuff
And In your $SPLUNK_HOME/etc/system/local/props.conf
[junpiper_syslog_stuff]
LINE_BREAKER=(\x00)<\d+>
SHOULD_LINEMERGE=False*

And changed the tcp to 1468 which is the port I am using. This does not work and I still get the lines added together. Looking at the actual log output in splunk I can see that the line break is different in my system \x00<133> but I have tried every possible permutation of that in the LINE_BREAKER expression and I cannot get it to work

I am sure I am just being a muppet but some help would be appreciated

25/10/2010 19:55:00.000

zone=Untrust dst zone=Trust action=Permit sent=887 rcvd=529 src=93.189.29.26 dst=212.21.101.220 src_port=52584 dst_port=80 src-xlated ip=93.189.29.26 port=52584 dst-xlated ip=212.21.101.220 port=80 session_id=1824 reason=Close - TCP FIN\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:56" duration=1 policy_id=6 service=http proto=6 src zone=Untrust dst zone=Trust action=Permit sent=1312 rcvd=12852 src=93.189.29.26 dst=212.21.101.220 src_port=52588 dst_port=80 src-xlated ip=93.189.29.26 port=52588 dst-xlated ip=212.21.101.220 port=80 session_id=3203 reason=Close - TCP FIN\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:53" duration=4 policy_id=9 service=dns proto=17 src zone=Untrust dst zone=Trust action=Permit sent=83 rcvd=215 src=195.96.0.4 dst=212.21.101.193 src_port=47092 dst_port=53 src-xlated ip=195.96.0.4 port=47092 dst-xlated ip=212.21.101.193 port=53 session_id=3973 reason=Close - RESP\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:55" duration=2 policy_id=6 service=http proto=6 src zone=Untrust dst zone=Trust action=Permit sent=3196 rcvd=897 src=88.97.218.190 dst=212.21.101.217 src_port=64015 dst_port=80 src-xlated ip=88.97.218.190 port=64015 dst-xlated ip=212.21.101.217 port=80 session_id=3983 reason=Close - TCP FIN\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:55" duration=2 policy_id=9 service=dns proto=17 src zone=Untrust dst zone=Trust action=Permit sent=97 rcvd=226 src=195.252.72.67 dst=212.21.101.193 src_port=32768 dst_port=53 src-xlated ip=195.252.72.67 port=32768 dst-xlated ip=212.21.101.193 port=53 session_id=2524 reason=Close - RESP\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:53" duration=4 policy_id=6 service=http proto=6 src zone=Untrust dst zone=Trust action=Permit sent=7156 rcvd=40647 src=79.173.154.37 dst=212.46.132.46 src_port=53796 dst_port=80 src-xlated ip=79.173.154.37 port=53796 dst-xlated ip=212.46.132.46 port=80 session_id=2075 reason=Close - TCP FIN\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:49" duration=8 policy_id=6 service=http proto=6 src zone=Untrust dst zone=Trust action=Permit sent=3679 rcvd=62167 src=79.173.154.37 dst=212.46.132.46 src_port=53792 dst_port=80 src-xlated ip=79.173.154.37 port=53792 dst-xlated ip=212.46.132.46 port=80 session_id=3941 reason=Close - TCP FIN\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:55" duration=2 policy_id=9 service=dns proto=17 src zone=Untrust dst zone=Trust action=Permit sent=97 rcvd=178 src=203.135.190.6 dst=212.21.101.193 src_port=5413 dst_port=53 src-xlated ip=203.135.190.6 port=5413 dst-xlated ip=212.21.101.193 port=53 session_id=2896 reason=Close - RESP\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:53" duration=4 policy_id=6 service=https proto=6 src zone=Untrust dst zone=Trust action=Permit sent=194 rcvd=130 src=217.147.95.3 dst=212.46.132.46 src_port=34187 dst_port=443 src-xlated ip=217.147.95.3 port=34187 dst-xlated ip=212.46.132.46 port=443 session_id=3287 reason=Close - AGE OUT\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:53" duration=4 policy_id=6 service=https proto=6 src zone=Untrust dst zone=Trust action=Permit sent=194 rcvd=130 src=217.147.95.3 dst=212.46.132.46 src_port=34184 dst_port=443 src-xlated ip=217.147.95.3 port=34184 dst-xlated ip=212.46.132.46 port=443 session_id=2261 reason=Close - AGE OUT\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:53" duration=4 policy_id=6 service=https proto=6 src zone=Untrust dst zone=Trust action=Permit sent=194 rcvd=130 src=217.147.95.3 dst=212.46.132.46 src_port=34179 dst_port=443 src-xlated ip=217.147.95.3 port=34179 dst-xlated ip=212.46.132.46 port=443 session_id=3654 reason=Close - AGE OUT\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:53" duration=4 policy_id=6 service=https proto=6 src zone=Untrust dst zone=Trust action=Permit sent=194 rcvd=130 src=217.147.95.3 dst=212.46.132.46 src_port=34178 dst_port=443 src-xlated ip=217.147.95.3 port=34178 dst-xlated ip=212.46.132.46 port=443 session_id=3466 reason=Close - AGE OUT\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:53" duration=4 policy_id=6 service=https proto=6 src zone=Untrust dst zone=Trust action=Permit sent=194 rcvd=130 src=217.147.95.3 dst=212.46.132.46 src_port=34173 dst_port=443 src-xlated ip=217.147.95.3 port=34173 dst-xlated ip=212.46.132.46 port=443 session_id=2486 reason=Close - AGE OUT\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:53" duration=4 policy_id=6 service=https proto=6 src zone=Untrust dst zone=Trust action=Permit sent=194 rcvd=130 src=217.147.95.3 dst=212.46.132.46 src_port=34165 dst_port=443 src-xlated ip=217.147.95.3 port=34165 dst-xlated ip=212.46.132.46 port=443 session_id=2716 reason=Close - AGE OUT\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:55" duration=2 policy_id=6 service=https proto=6 src zone=Untrust dst zone=Trust action=Permit sent=194 rcvd=130 src=217.147.95.3 dst=212.46.132.46 src_port=34254 dst_port=443 src-xlated ip=217.147.95.3 port=34254 dst-xlated ip=212.46.132.46 port=443 session_id=4043 reason=Close - AGE OUT\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:54" duration=3 policy_id=6 service=https proto=6 src zone=Untrust dst zone=Trust action=Permit sent=194 rcvd=130 src=217.147.95.3 dst=212.46.132.46 src_port=34249 dst_port=443 src-xlated ip=217.147.95.3 port=34249 dst-xlated ip=212.46.132.46 port=443 session_id=2318 reason=Close - AGE OUT\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:54" duration=3 policy_id=6 service=https proto=6 src zone=Untrust dst zone=Trust action=Permit sent=194 rcvd=130 src=217.147.95.3 dst=212.46.132.46 src_port=34248 dst_port=443 src-xlated ip=217.147.95.3 port=34248 dst-xlated ip=212.46.132.46 port=443 session_id=2625 reason=Close - AGE OUT\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:54" duration=3 policy_id=6 service=https proto=6 src zone=Untrust dst zone=Trust action=Permit sent=194 rcvd=130 src=217.147.95.3 dst=212.46.132.46 src_port=34241 dst_port=443 src-xlated ip=217.147.95.3 port=34241 dst-xlated ip=212.46.132.46 port=443 session_id=2467 reason=Close - AGE OUT\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:54" duration=3 policy_id=6 service=https proto=6 src zone=Untrust dst zone=Trust action=Permit sent=194 rcvd=130 src=217.147.95.3 dst=212.46.132.46 src_port=34226 dst_port=443 src-xlated ip=217.147.95.3 port=34226 dst-xlated ip=212.46.132.46 port=443 session_id=3831 reason=Close - AGE OUT\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:54" duration=3 policy_id=6 service=https proto=6 src zone=Untrust dst zone=Trust action=Permit sent=194 rcvd=130 src=217.147.95.3 dst=212.46.132.46 src_port=34218 dst_port=443 src-xlated ip=217.147.95.3 port=34218 dst-xlated ip=212.46.132.46 port=443 session_id=2672 reason=Close - AGE OUT\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:54" duration=3 policy_id=6 service=https proto=6 src zone=Untrust dst zone=Trust action=Permit sent=194 rcvd=130 src=217.147.95.3 dst=212.46.132.46 src_port=34210 dst_port=443 src-xlated ip=217.147.95.3 port=34210 dst-xlated ip=212.46.132.46 port=443 session_id=2063 reason=Close - AGE OUT\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:53" duration=4 policy_id=6 service=https proto=6 src zone=Untrust dst zone=Trust action=Permit sent=194 rcvd=130 src=217.147.95.3 dst=212.46.132.46 src_port=34203 dst_port=443 src-xlated ip=217.147.95.3 port=34203 dst-xlated ip=212.46.132.46 port=443 session_id=3326 reason=Close - AGE OUT\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:53" duration=4 policy_id=6 service=https proto=6 src zone=Untrust dst zone=Trust action=Permit sent=194 rcvd=130 src=217.147.95.3 dst=212.46.132.46 src_port=34197 dst_port=443 src-xlated ip=217.147.95.3 port=34197 dst-xlated ip=212.46.132.46 port=443 session_id=3637 reason=Close - AGE OUT\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time="2010-10-25 17:19:53" duration=4 policy_id=6 service=https proto=6 src zone=Untrust dst zone=Trust action=Permit sent=194 rcvd=130 src=217.147.95.3 dst=212.46.132.46 src_port=34194 dst_port=443 src-xlated ip=217.147.95.3 port=34194 dst-xlated ip=212.46.132.46 port=443 session_id=3175 reason=Close - AGE OUT\x00<133>ssg5-serial: NetScreen device_id=0162102007000604 [Root]system-notification-00257(traffic): start_time=

Tags (2)
0 Karma

dwaddle
SplunkTrust
SplunkTrust

The <133> is the syslog "facility" and "level" encoded into the message. The "\x00" is probably a null termination at the end of each message. One of my questions would be is the "\x00" actually a 0x00 byte, or the bytes 0x5C 0x78 0x30 0x30? A wireshark capture would tell you for sure.

Once you know, you could update the LINE_BREAKER to eat it as well, or use a SEDCMD in props.conf to filter it out.

Also, the <133> may not always have the digits "133" in it. Generalizing your regexp to 1 to 3 digits inside the <>'s would make it work if the Netscreen sends a different syslog level for some reason.

LINE_BREAKER=(<\d{1,3}>)

Dragonnet
New Member

Sorted.

LINE_BREAKER=(<133>)

Works just fine, It does leave an entry of \x100 behind every reason= field and one day I will work out how to get rid of that.

If anyone can tell me how to kill that I would be grateful

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...