How come I'm unable to extract multiline events with the multikv command?

[carao2020]

Hi,

I am trying to extract events from multiline event using multikv. Could someone please help me in configuring the multikv for these kind of events?

[tiagofbmm]

The approach should be breaking those events at Indextime, not at search time. You should have your sourcetype with the following parameters:

SHOULD_LINEMERGE=true
LINE_BREAKER=([\r\n]+)
NO_BINARY_CHECK=true
BREAK_ONLY_BEFORE=App
TIME_FORMAT=%d/%m/%Y %H:%:%M:%S.%3N
MAX_TIMESTAMP_LOOKAHEAD=40
TRUNCATE=1000

That way your events will be well defined, check the image below

[carao2020]

Hi Tiagofbmm,

Thanks for the details. The actual event payload is huge XML file and I had broken down that XML event and tried extracting this event content from XML. Could you please review and suggest if we can split this by multikv conf file.. Thanks in advance.. Additionally could you please suggest whether index time event breaking is good for performance than searchtime event breaking. Thanks in advance

[tiagofbmm]

If the XML is big you can always increase the TRUNCATE parameter to accommodate that. multikv doesn't seem to apply to what you have: every line of your data is not an event. YOur scenario is clearly dependent on proper event breaking settings like I showed you. Can you try using these:

SHOULD_LINEMERGE=true
LINE_BREAKER=([\r\n]+)
NO_BINARY_CHECK=true
BREAK_ONLY_BEFORE=App
TIME_FORMAT=%d/%m/%Y %H:%:%M:%S.%3N
MAX_TIMESTAMP_LOOKAHEAD=40
TRUNCATE=100000

[tiagofbmm]

https://docs.splunk.com/Documentation/Splunk/7.0.2/SearchReference/Multikv

What exactly is the problem?

[carao2020]

Hi Tiagofbmm,

Thanks for the response. Please see below sample event data that I am trying to extract the events using multikv command. I need to create one event for each chunk of data starts with "App - Error:" and Endswith "Solution: " so that I can extract the fields in subsequent stages. Kindly review and suggest configuration for multikv.conf.

<![CDATA[
Sample Software Group, Inc. cerbnav0
Batch Router Module
Version 123.45678.91011.1001 Oct 20 2017 15:41:03
© 2017 Sample Software Group, Inc. All rights reserved. 

Application Server: SAMPLEHOST
Process ID        : 7392
User              : sample_user

*-- Standard Error --*
================================
Initialization Phase Started.

Actual Run Start Date and Time     [01/22/2019 20:09:02.970]

Initialization Phase Complete.
================================
================================
Execution Phase Started.


Sample Software Group, Inc. cmcbclmu
Claims Multi-Eng Electronic Adjud. Program
Version 123.45678.91011.1001 Oct 20 2017 16:00:40
© 2017 Sample Software Group, Inc. All rights reserved. 

Application Server: SAMPLEHOST
Process ID        : 7392
User              : sample_user

*-- Standard Error --*
================================
Initialization Phase Started.

Actual Run Start Date and Time     [01/22/2019 20:09:02.970]

Initialization Phase Complete.
================================
================================
Execution Phase Started.

App - Error: 01/22/2019 20:09:24.530
Return Code: 8
Error Code: 51301
Error Message: ABC697868600 Pended due to Security Violation. Warning Message limit exceeds User limit
Warn Number = 710 Warning Text =  COBZ - Other Carrier Paid is Zero  (line 1)
Pend Reason =  COBZ WMUD
Solution:  

App - Error: 01/22/2019 20:11:47.646
Return Code: 8
Error Code: 51301
Error Message: ABCDE4538400 Pended due to Security Violation. Warning Message limit exceeds User limit
Warn Number = 710 Warning Text =  MNEC Review Denied Services for Medical Necessity  (line 3)
Pend Reason =  MNEC WMUD
Solution:  

App - Error: 01/22/2019 20:12:20.889
Return Code: 8
Error Code: 51301
Error Message: ABC683235200 Pended due to Security Violation. Warning Message limit exceeds User limit
Warn Number = 17 Warning Text =  ATHM Multiple UM pre-authorizations match on line 1
Pend Reason =  ATHM WMWM
Solution: