Line break involving line of ============

[gnovak]

I'm having trouble with a line breaking issue. I'm using the preview feature with splunk and I'm trying to get the line breaks to work properly. Currently splunk is breaking every time there is a timestamp and I don't want that. I need my log entries to look like this and be one event for each chunk of data:

    ===========================================
    Check start: Tue Feb  5 23:28:10 EST 2013
    ===========================================
    Accessing ATOMPUB_URL: http://BLAHBLAH:810/wackadoo/lobster with userid admin@AWESOMEHOST
    Grabbing heartbeat folder...
    Reading test file for upload...
    Checking if file already exists...
    Creating test upload...
      Document successfully created.
    Verifying that document was successfully uploaded...
    Verifying that rendition was successfully created...
    Deleting uploaded document...
       Deletion successful.
    Test successfully completed.
    Heartbeat check returned in 4001ms.
    Heartbeat check status:0
    ===========================================
    Check end: Tue Feb  5 23:28:14 EST 2013
    ===========================================

Splunk is currently breaking up this one event into two events that look like this:

Event 1

Check start: Tue Feb  5 23:28:10 EST 2013
    ===========================================
    Accessing ATOMPUB_URL: http://BLAHBLAH:810/wackadoo/lobster with userid admin@AWESOMEHOST
    Grabbing heartbeat folder...
    Reading test file for upload...
    Checking if file already exists...
    Creating test upload...
      Document successfully created.
    Verifying that document was successfully uploaded...
    Verifying that rendition was successfully created...
    Deleting uploaded document...
       Deletion successful.
    Test successfully completed.
    Heartbeat check returned in 4001ms.
    Heartbeat check status:0
    ===========================================

Event 2

    Check end: Tue Feb  5 23:26:20 EST 2013
===========================================
===========================================

I need to make it look like first entry I posted and have it be 1 event.

So far I have Timestamp is always prefaced by a pattern and this regex:
\bCheck \w+: \w+ \w+ \d+ \d+:\d+:\d+ \w+ \d+\b

I have this for my props.conf:
MUST_BREAK_AFTER = \bCheck end: \w+ \w+ \d+ \d+:\d+:\d+ \w+ \d+\b

It's almost there, but I still am having issues with a line of =======================. Splunk now see's all of this as one event with these settings:

===========================================
===========================================
Check start: Tue Feb  5 23:31:59 EST 2013
===========================================
Accessing ATOMPUB_URL: http://blahblah:810/booberry/lobster with userid admin@AWESOME
Grabbing heartbeat folder...
Reading test file for upload...
Checking if file already exists...
Creating test upload...
  Document successfully created.
Verifying that document was successfully uploaded...
Verifying that rendition was successfully created...
Deleting uploaded document...
   Deletion successful.
Test successfully completed.
Heartbeat check returned in 3501ms.
Heartbeat check status:0
===========================================
Check end: Tue Feb  5 23:32:03 EST 2013

I also tried specifying the timestamp format with %a %b %H:%M:%S %Z %Y for the timestamp. I kept getting the message that it "failed to parse timestamp" and was defaulting to the file modtime. This didn't do much to get rid of this error.

Any ideas how to get that last ================= under Check end? If not I'll probably just do without but was curious if there was something I missed.

[martin_mueller]

This may look ugly, but it should work:

LINE_BREAKER=(?s)([\n\r]+)(.{45}Check start)

Anything else should be able to remain with default values. For some reason my previewer didn't accept the equal sign in the RHS...

[jeff]

a similar answer, but closer to the actual expected output

LINE_BREAKER=([\r\n]+)(=+[\r\n]+Check start)
SHOULD_LINEMERGE=false

should throw out the first [\r\n]+, and begin each event with your first row of =+

[gnovak]

I'm really having no luck with the preview feature of splunk. I tried this, and it just jumbled everything into one huge giant event. I did have my file look nice in preview, but when I setup the sourcetype and it put it in props.conf, the file in splunk looked nothing like it did in preview. I'm not sure what the problem is. IN preview this one line worked to get me what I want actually: BREAK_ONLY_BEFORE = \bCheck start: \w+ \w+ \d+ \d+:\d+:\d+ \w+ \d+\b. However once implemented, the log did not look like this.