Hi Splunkers,
I got the following multi-line event.
# Time: 150601 17:30:31
# User@Host: sample[sample] @ SAMPLEAP [192.168.1.3]
# Query_time: 3.295358 Lock_time: 0.000139 Rows_sent: 271 Rows_examined: 229938
SET timestamp=1433147431;
SELECT YYYYYYYYYYYYYYYYYYYYYYYY
# Time: 150601 17:30:32
# User@Host: sample[sample] @ SAMPLEAP [192.168.1.3]
# Query_time: 3.278466 Lock_time: 0.000115 Rows_sent: 271 Rows_examined: 229942
SET timestamp=1433147432;
SELECT ZZZZZZZZZZZZZZZZZZZZZZZZ
# User@Host: sample[sample] @ SAMPLEAP [192.168.1.3]
# Query_time: 3.487538 Lock_time: 0.000079 Rows_sent: 271 Rows_examined: 229942
SET timestamp=1433147432;
INSERT BBBBBBBBBBBBBBBBBBBBBBBB
I want to split this log into three events as followings. A "Time" record need to be always top of each event if it exists.
But if it doesn't exist, event need to be split to a event beginning with a "User" record.
# Time: 150601 17:30:31
# User@Host: sample[sample] @ SAMPLEAP [192.168.1.3]
# Query_time: 3.295358 Lock_time: 0.000139 Rows_sent: 271 Rows_examined: 229938
SET timestamp=1433147431;
SELECT YYYYYYYYYYYYYYYYYYYYYYYY
# Time: 150601 17:30:32
# User@Host: sample[sample] @ SAMPLEAP [192.168.1.3]
# Query_time: 3.278466 Lock_time: 0.000115 Rows_sent: 271 Rows_examined: 229942
SET timestamp=1433147432;
SELECT ZZZZZZZZZZZZZZZZZZZZZZZZ
# User@Host: sample[sample] @ SAMPLEAP [192.168.1.3]
# Query_time: 3.487538 Lock_time: 0.000079 Rows_sent: 271 Rows_examined: 229942
SET timestamp=1433147432;
INSERT BBBBBBBBBBBBBBBBBBBBBBBB
How to make it ?
Thank you for your help.
Well, BREAK_ONLY_BEFORE is not going to help here. Try this in the appropriate props.conf
[yoursourcetypehere]
MUST_BREAK_AFTER = ^\sINSERT
BREAK_ONLY_BEFORE_DATE = false
MAX_TIMESTAMP_LOOKAHEAD = 20
TIME_PREFIX = \#\sTime:\s
It looks like there is a space at the beginning of the line before the INSERT, so I included that in the regular expression above. Do you want the Splunk timestamp to be based on the Time (if it exists) or on the timestamp=ddddddd? The settings above are for the Time entry. If you want the timestamp, use the two lines below instead:
MAX_TIMESTAMP_LOOKAHEAD = 10
TIME_PREFIX = timestamp\=
HTH
Well, BREAK_ONLY_BEFORE is not going to help here. Try this in the appropriate props.conf
[yoursourcetypehere]
MUST_BREAK_AFTER = ^\sINSERT
BREAK_ONLY_BEFORE_DATE = false
MAX_TIMESTAMP_LOOKAHEAD = 20
TIME_PREFIX = \#\sTime:\s
It looks like there is a space at the beginning of the line before the INSERT, so I included that in the regular expression above. Do you want the Splunk timestamp to be based on the Time (if it exists) or on the timestamp=ddddddd? The settings above are for the Time entry. If you want the timestamp, use the two lines below instead:
MAX_TIMESTAMP_LOOKAHEAD = 10
TIME_PREFIX = timestamp\=
HTH
TIME_PREFIX = \#\sTime:\s
MAX_TIMESTAMP_LOOKAHEAD = 20
Thank you very much, Iguinn. These attributes are very helpful to me. By the way, the space at the beginning of the line are the specification of this Answers site, original log doesn't include any space at the beginning. But there is a space when I paste it in this site.
In that case, the MUST_BREAK_AFTER
should look like this
MUST_BREAK_AFTER = ^INSERT