I am trying to analyze exception logging that is written across multiple lines, and extract only certain lines of the event into fields. I have been reading documentation and posts which seem to suggest defining stanzas in transforms.conf and props.conf would be the preferred way to tackle this. I have tried to implement to no avail and am lost! I would appreciate any ideas/suggestions on how to properly implement!!
Here are a few example log snippets to help demonstrate the challenge. Given the following, I need to extract...
(a) exception message (in bold/italics)
(b) the first calling method from either SOURCE_B or SOURCE_C, but not SOURCE_A
(1)
20160825 12:51:16 unhandled error from dispatcher, sender:System.Windows.Threading.Dispatcher
System.NullReferenceException: Object reference not set to an instance of an object.
at SOURCE_A.Method(Object sender, ExecutedRoutedEventArgs e)
at SOURCE_A.Method(Object sender, ExecutedRoutedEventArgs e)
at SOURCE_B.Method(Object sender, ExecutedRoutedEventArgs e, CommandBinding commandBinding)
(2)
20160825 12:53:16 unhandled error from dispatcher, sender:System.Windows.Threading.Dispatcher
System.Runtime.InteropServices.COMException ().
at SOURCE_C.Method(FORMATETC& format, STGMEDIUM& medium)
with the following results
(1)
UE_msg: System.NullReferenceException: Object reference not set to an instance of an object.
UE_method: SOURCE_B.Method(Object sender, ExecutedRoutedEventArgs e, CommandBinding commandBinding)
(2)
UE_msg: System.Runtime.InteropServices.COMException ()
UE_method: SOURCE_C.Method(FORMATETC& format, STGMEDIUM& medium)
The logging is not very structured...but all of these exceptions include the "unhandled error" string pattern, with the high-level "exception message" following on the next line that I need to extract, then SOURCE_B or SOURCE_C methods following below that somewhere in the stacktrace. So my thoughts are to define a REGEX stanza in transforms.config as follows
(transforms.config)
[UE_regex]
REGEX = (?m)(unhandled error.\*\n)(.\*\\.)((SOURCE_B|SOURCE_C).*\\))
FORMAT = UE_msg::$2 UE_method::$3
(props.config)
[UE]
REPORT-UE = UE_regex
lastly...try to table results....
source="c:\\logs\\perf*" sourcetype="UE" | table results
I have been tweaking the REGEX patterns, groupings, tried ditching transforms.config and tried defining just an EXTRACT in props.config. But nothing has yielded any results. At this point I can't tell if I'm even on the correct path anymore and would appreciate some guidance!
thanks!!!
Try separate REPORT. Like this
*props*
[UE]
REPORT-UE = UE_msg
REPORT-UE = UE_method
*transforms*
[UE_msg]
REGEX = (?ms)unhandled error.*\n(?<UE_msg>[^\n]+)
[UE_method]
REGEX = at\s+(?<UE_method>SOURCE_[B|C])
Try separate REPORT. Like this
*props*
[UE]
REPORT-UE = UE_msg
REPORT-UE = UE_method
*transforms*
[UE_msg]
REGEX = (?ms)unhandled error.*\n(?<UE_msg>[^\n]+)
[UE_method]
REGEX = at\s+(?<UE_method>SOURCE_[B|C])
thank you sundareshr - this got me on the right track.
I tried the separate report, but it was only taking the last-assign stanza. i.e. UE_method was extracted but not UE_msg.
Continuing with this approach however, the following seems to work:
props.conf
REPORT-UEmsg = UE_msg
REPORT-UEmethod = UE_method
transforms.conf
[UE_msg]
REGEX = (?m)unhandled error.*(?System\D+\:.*)\.\s
[UE_method]
REGEX = (?m)unhandled error.*(?(SOURCE_B|SOURCE_C)\D+)\s
this has gotten me very close to what I need. I still am wrestling with the regex of the UE_method as it is extracting the rest of the stack trace instead of the specific line I want, but I will other posts for a solution and post a new one if needed.
thanks!