We have a multi line message that looks like this:
11/30/10 16:28:34 Verifying pricing env CLOSE,FX_CLOSE,XLA_ENV,INTRADAY,CPTY_CREDIT
No exceptions for CLOSE, loaded in 0.05 secs
Messages for FX_CLOSE
PricerConfigRefresh: No item found for 1246892/CurveZero (CLOSE) before Tue Nov 30 16:28:34 EST 2010
No exceptions for FX_CLOSE, loaded in 0.17 secs
No exceptions for XLA_ENV, loaded in 0.05 secs
No exceptions for INTRADAY, loaded in 0.10 secs
No exceptions for CPTY_CREDIT, loaded in 0.40 sec
I would like to create the following fields and assign the numeric values as response time, so I can search and chart on the RT values.
close_pe_rt=0.05
fx_close_pe_rt=0.17
xla_env_pe_rt=0.05
intraday_pe_rt=0.10
cpty_credit_pe_rt=0.40
What is the best way to handle such extraction and assignments?
Thanks very much for your help, Jean
These are the 2 options I would try
The easiest, but also most transient, option is to use rex command inline in your search. For example:
sourcetype="multiline" | rex "CLOSE, loaded in (?<close_pe_rt>\S+)" | rex "FX_CLOSE, loaded in (?<fx_close_pe_rt>\S+)" | rex "XLA_ENV, loaded in (?<xla_env_pe_rt>\S+)" | rex "INTRADAY, loaded in (?<intraday_pe_rt>\S+)" | rex "CPTY_CREDIT, loaded in (?<cpty_credit_pe_rt>\S+)"
Maybe there's a way to do this in one rex invocation, but I tried several things which didn't work.
The other option is to add a few stanzas to props.conf and transforms.conf. For example,
in props.conf:
[multiline]
REPORT-foo = mlFields
in transforms.conf:
[mlFields]
REGEX = CLOSE, loaded in (\S+).* FX_CLOSE, loaded in (\S+).* XLA_ENV, loaded in (\S+).* INTRADAY, loaded in (\S+).* CPTY_CREDIT, loaded in (\S+)
FORMAT = close_pe_rt::$1 fx_close_pe_rt::$2 xla_env_pe_rt::$3 intraday_pe_rt::$4 cpty_credit_pe_rt::$5
You could also try using the Interactive Field Extractor (IFX).
These are the 2 options I would try
The easiest, but also most transient, option is to use rex command inline in your search. For example:
sourcetype="multiline" | rex "CLOSE, loaded in (?<close_pe_rt>\S+)" | rex "FX_CLOSE, loaded in (?<fx_close_pe_rt>\S+)" | rex "XLA_ENV, loaded in (?<xla_env_pe_rt>\S+)" | rex "INTRADAY, loaded in (?<intraday_pe_rt>\S+)" | rex "CPTY_CREDIT, loaded in (?<cpty_credit_pe_rt>\S+)"
Maybe there's a way to do this in one rex invocation, but I tried several things which didn't work.
The other option is to add a few stanzas to props.conf and transforms.conf. For example,
in props.conf:
[multiline]
REPORT-foo = mlFields
in transforms.conf:
[mlFields]
REGEX = CLOSE, loaded in (\S+).* FX_CLOSE, loaded in (\S+).* XLA_ENV, loaded in (\S+).* INTRADAY, loaded in (\S+).* CPTY_CREDIT, loaded in (\S+)
FORMAT = close_pe_rt::$1 fx_close_pe_rt::$2 xla_env_pe_rt::$3 intraday_pe_rt::$4 cpty_credit_pe_rt::$5
You could also try using the Interactive Field Extractor (IFX).
Thank you very much.
This solution worked quite well and I implemented it with the props.conf and transforms.conf.
Note: I did not have to use the (?m) regex modifier in the REGEX field for transforms.conf. Somewhere along the way, Splunk automatically knows how to deal with multiline events.
You can either use multiple regular expressions (multiple rex command calls or multiple EXTRACT-* options in props.conf) or a single regex with the (?m)
multiline flag (maybe as well the (?s)
DOTALL flag).
Someting like this might work:
... | rex "(?ms)No exceptions for CLOSE, loaded in (?<close_pe_rt>[\d\.]+).+FX_CLOSE, loaded in (?<fx_close_pe_rt>[\d\.]+).+XLA_ENV, loaded in (?<xla_env_pe_rt>[\d\.]+).+INTRADAY, loaded in (?<intraday_pe_rt>[\d\.]+).+CPTY_CREDIT, loaded in (?<cpty_credit_pe_rt>[\d\.]+)"