Splunk Search

How to configure line breaking for mixed single line and multiline logs?

jamesvz84
Communicator

Given the following log format, is it possible to store the consecutive GROUPED/GROUPED_DET lines into one event while leaving LOGTYPE1 & LOGTYPE2 lines as singled-lined events? One of my main issues is also trying to come up with a regex for the LINE_BREAKER that will ONLY break if the line does NOT contain GROUPED : or GROUPED_DET:. Seems like Splunk will only look for a positive condition (break if x exists) rather than a negative condition (break if line does NOT contain x). Is that the case, or is there a regex or other settings that will allow proper breaking on this log?

LOGTYPE1 :2015/02/28 18:13:04.253|2015/02/28|118:13:04.252|6|LOC|4|0.01||||||||||||||0|50|41.25
LOGTYPE1 :2015/02/28 18:13:26.002|2015/02/28|18:13:26.002|5|GPS|3|1|39:48.0219|N|105:4.0201|W|2|10|1.0
GROUPED :2015/02/28 18:13:57.591:GROUP:Rcvd Office msg [1080] from AG
GROUPED_DET:2015/02/28 18:13:57.591:GROUP:000 | 04 04 38 01 11 00 00 09 00 03 d1 75 54 f2 05 66 | ..8........uT..f
GROUPED_DET:2015/02/28 18:13:57.592:GROUP:001 | 28 00 37 00 00 62 6e 73 66 2e 62 3a 70 74 63 73 | (.7..xxxx.b:ptcs
GROUPED_DET:2015/02/28 18:13:57.592:GROUP:002 | 72 76 72 00 62 6e 73 66 2e 6c 2e 62 6e 73 66 2e | rvr.xxxx.l.xxxx.
GROUPED_DET:2015/02/28 18:13:57.592:GROUP:003 | 34 34 31 37 3a 69 74 63 00 42 4e 53 46 01 00 00 | 4417:itc.XXXX...
GROUPED_DET:2015/02/28 18:13:57.592:GROUP:004 | 00 00 7d a3 6b f9                               | ..}.k.
LOGTYPE2 :2015/02/28 18:13:57.595|2015/02/28|18:13:57.594|21|EMP|2|1080|1|1|04043801110000090003d17554f205662800370000626e73662e623a7074637372767200626e73662e6c2e626e73662e343431373a69746300424e534601000000007da36bf9
GROUPED :2015/02/28 18:13:58.546:GROUP:Sending Office msg [2080]
GROUPED_DET:2015/02/28 18:13:58.546:GROUP:000 | 04 08 20 04 11 00 00 3b 00 03 d1 75 54 f2 05 66 | .. ....;...uT..f
GROUPED_DET:2015/02/28 18:13:58.546:GROUP:001 | 28 01 2c 00 20 62 6e 73 66 2e 6c 2e 62 6e 73 66 | (.,. xxxx.l.xxxx
GROUPED_DET:2015/02/28 18:13:58.546:GROUP:002 | 2e 34 34 31 37 3a 69 74 63 00 62 6e 73 66 2e 62 | .4417:itc.xxxx.b
GROUPED_DET:2015/02/28 18:13:58.546:GROUP:003 | 3a 70 74 63 73 72 76 72 00 42 4e 53 46 00 00 00 | :ptcsrvr.XXXX...
GROUPED_DET:2015/02/28 18:13:58.546:GROUP:004 | 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 | ................
GROUPED_DET:2015/02/28 18:13:58.547:GROUP:005 | 00 00 00 00 00 00 00 00 00 00 00 00 00 a4 00 f8 | ................
GROUPED_DET:2015/02/28 18:13:58.547:GROUP:006 | 65 2e 0d e3 c0 35 12 18 36 21 7b 02 00 02 04 01 | e....5..6!{.....
GROUPED_DET:2015/02/28 18:13:58.547:GROUP:007 | 00 00 00 00 cc da af de                         | ........
LOGTYPE1 :2015/02/28 18:13:58.550|2015/02/28|18:13:58.550|21|EMP|2|2080|1|1|040820041100003b0003d17554f2056628012c0020626e73662e6c2e626e73662e343431373a69746300626e73662e623a7074637372767200424e53460000000000000000000000000000000000000000000000000000000000000000a400f8652e0de3c035121836217b
0 Karma
1 Solution

jamesvz84
Communicator

The following worked (uses negative lookahead regex):

[sourcetype]
SHOULD_LINEMERGE = true 
LINE_BREAKER = ([\r\n]+)(?!GROUPED_DET)

View solution in original post

jamesvz84
Communicator

The following worked (uses negative lookahead regex):

[sourcetype]
SHOULD_LINEMERGE = true 
LINE_BREAKER = ([\r\n]+)(?!GROUPED_DET)
Get Updates on the Splunk Community!

Introducing the Splunk Community Dashboard Challenge!

Welcome to Splunk Community Dashboard Challenge! This is your chance to showcase your skills in creating ...

Built-in Service Level Objectives Management to Bridge the Gap Between Service & ...

Wednesday, May 29, 2024  |  11AM PST / 2PM ESTRegister now and join us to learn more about how you can ...

Get Your Exclusive Splunk Certified Cybersecurity Defense Engineer Certification at ...

We’re excited to announce a new Splunk certification exam being released at .conf24! If you’re headed to Vegas ...