Getting Data In

Line break with multiple Linebreaker

ss026381
Communicator

I have a log file something like this

PStart  bla bla bla bla 
PQL% bla bla bla bla
PEnd bla bla bla bla 
PQL% bla bla bla bla
bla bla blabla bla bla
PQL% bla bla bla bla

I want to break it at PStart or PQL% but if PQL% is within PStart and PEnd keep it one event.

For the above example, I want to have 3 events as follow

Event 1

PStart  bla bla bla bla 
PQL% bla bla bla bla
PEnd bla bla bla bla 

Event 2

PQL% bla bla bla bla
bla bla blabla bla bla

Event 3

PQL% bla bla bla bla

The code I have is following which doesn't work properly

[sourcetype]
LINE_BREAKER = ([\r\n]+)(?:(?:PStart|(?:PQL%))) 
MUST_NOT_BREAK_BEFORE = ([\r\n]+)(?:PEnd) 
Tags (2)
1 Solution

acharlieh
Influencer

With the sample as given this is what I come up with:

[sourcetype]
LINE_BREAKER=([\r\n]+)(?:PStart|PEnd|PQL%)
SHOULD_LINEMERGE=true
BREAK_ONLY_BEFORE=^PStart|^PQL%
MUST_NOT_BREAK_AFTER=^PStart
MUST_BREAK_AFTER=^PEnd

First during the line breaking stage, we break up into "lines" that start PStart, PEnd or PQL% (This is only necessary if a PEnd line may contain carrage returns or line feeds, otherwise you can get away with the default ([\r\n]+) here

Then during line merging, we only break events before a PStart or PQL% line... this gets us our rough separation of events... but then with the MUST_NOT_BREAK_AFTER / MUST_BREAK_AFTER combination, we can ensure that from PStart to PEnd is a single block. (Obviously does not support embedding, but works single level).

View solution in original post

acharlieh
Influencer

With the sample as given this is what I come up with:

[sourcetype]
LINE_BREAKER=([\r\n]+)(?:PStart|PEnd|PQL%)
SHOULD_LINEMERGE=true
BREAK_ONLY_BEFORE=^PStart|^PQL%
MUST_NOT_BREAK_AFTER=^PStart
MUST_BREAK_AFTER=^PEnd

First during the line breaking stage, we break up into "lines" that start PStart, PEnd or PQL% (This is only necessary if a PEnd line may contain carrage returns or line feeds, otherwise you can get away with the default ([\r\n]+) here

Then during line merging, we only break events before a PStart or PQL% line... this gets us our rough separation of events... but then with the MUST_NOT_BREAK_AFTER / MUST_BREAK_AFTER combination, we can ensure that from PStart to PEnd is a single block. (Obviously does not support embedding, but works single level).

ss026381
Communicator

This works, thanks...

0 Karma

woodcock
Esteemed Legend

Be sure to click Accept to close this question and reward the author of this MOST TERRIBLY EXCELLENT ANSWER!!!!

0 Karma

ss026381
Communicator

Thank you guys, I accepted, special thanks to Charlie.

0 Karma

woodcock
Esteemed Legend

This is a tough one. I am curious to see if there is a splunk-based solution. I've got nothing because negative-lookbehinds must be fixed-length.

0 Karma
Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...