How to configure BREAK_ONLY_BEFORE to split multiline event ?

[sunrise]

Hi Splunkers,

I got the following multi-line event.

# Time: 150601 17:30:31
# User@Host: sample[sample] @ SAMPLEAP [192.168.1.3]
# Query_time: 3.295358  Lock_time: 0.000139 Rows_sent: 271  Rows_examined: 229938
SET timestamp=1433147431;
SELECT YYYYYYYYYYYYYYYYYYYYYYYY
# Time: 150601 17:30:32
# User@Host: sample[sample] @ SAMPLEAP [192.168.1.3]
# Query_time: 3.278466  Lock_time: 0.000115 Rows_sent: 271  Rows_examined: 229942
SET timestamp=1433147432;
SELECT ZZZZZZZZZZZZZZZZZZZZZZZZ
# User@Host: sample[sample] @ SAMPLEAP [192.168.1.3]
# Query_time: 3.487538  Lock_time: 0.000079 Rows_sent: 271  Rows_examined: 229942
SET timestamp=1433147432;
INSERT BBBBBBBBBBBBBBBBBBBBBBBB

I want to split this log into three events as followings. A "Time" record need to be always top of each event if it exists.
But if it doesn't exist, event need to be split to a event beginning with a "User" record.

# Time: 150601 17:30:31
# User@Host: sample[sample] @ SAMPLEAP [192.168.1.3]
# Query_time: 3.295358  Lock_time: 0.000139 Rows_sent: 271  Rows_examined: 229938
SET timestamp=1433147431;
SELECT YYYYYYYYYYYYYYYYYYYYYYYY

# Time: 150601 17:30:32
# User@Host: sample[sample] @ SAMPLEAP [192.168.1.3]
# Query_time: 3.278466  Lock_time: 0.000115 Rows_sent: 271  Rows_examined: 229942
SET timestamp=1433147432;
SELECT ZZZZZZZZZZZZZZZZZZZZZZZZ

# User@Host: sample[sample] @ SAMPLEAP [192.168.1.3]
# Query_time: 3.487538  Lock_time: 0.000079 Rows_sent: 271  Rows_examined: 229942
SET timestamp=1433147432;
INSERT BBBBBBBBBBBBBBBBBBBBBBBB

How to make it ?
Thank you for your help.

[lguinn2 · Accepted solution]

Well, BREAK_ONLY_BEFORE is not going to help here. Try this in the appropriate props.conf

[yoursourcetypehere]
MUST_BREAK_AFTER = ^\sINSERT
BREAK_ONLY_BEFORE_DATE = false
MAX_TIMESTAMP_LOOKAHEAD = 20
TIME_PREFIX = \#\sTime:\s

It looks like there is a space at the beginning of the line before the INSERT, so I included that in the regular expression above. Do you want the Splunk timestamp to be based on the Time (if it exists) or on the timestamp=ddddddd? The settings above are for the Time entry. If you want the timestamp, use the two lines below instead:

MAX_TIMESTAMP_LOOKAHEAD = 10
TIME_PREFIX = timestamp\=

HTH

[sunrise]

TIME_PREFIX = \#\sTime:\s
MAX_TIMESTAMP_LOOKAHEAD = 20

Thank you very much, Iguinn. These attributes are very helpful to me. By the way, the space at the beginning of the line are the specification of this Answers site, original log doesn't include any space at the beginning. But there is a space when I paste it in this site.

[lguinn2]

In that case, the MUST_BREAK_AFTER should look like this

MUST_BREAK_AFTER = ^INSERT