I have created a regex that works fine during search time, but when added to props.conf and/or transforms.conf to extract the field during index time, the field doesnt get extracted?
I dont understand how this could work during search time in the Splunk Search bar search page, but not when added to props.conf?
Here it is:
rex field=_raw "set=(?<phoneid>.+)\snotTime"
^(?:.+?,){4}(?
please stop posting comments as new answers. thanks.
i tried with the props only and i still cannot see the fields. has this anything to do with splunk 6.0.3. the other colleague of mine created field extraction and does not see them as well. it was ok two weeks ago before upgrade to splunk 6.0.3
props.conf
[sdf_bpel_metric]
REPORT-sdf_policy_metric = SDFCorepolicymetrics
transforms.conf
[SDFCorepolicymetrics]
REGEX = ^(?:.+?,){4}(?
props.conf only
EXTRACT-SDFCorepolicymetrics = ^(?:.+?,){4}(?
From here: http://regex101.com/r/eM2hB7
requestApplicationLabel [40-52] MetricLogger
requestTransactionID [53-67] TDI_CLOUDCSX_1
callingApplication [82-115] hymlxsdfbpe11_1401889362113_11537
callType [116-116] ``
function [117-140] RetrieveIdentityDetails
second log
requestApplicationLabel [63-75] MetricLogger
requestTransactionID [76-111] TELSTRA_PREPAIDACTIVATION_STRATEGIC
callingApplication [148-180] chslxsdfbpe05_1401889356427_2871
callType [181-181] ``
function [182-212] CCandB.CreateNewBillingAccount
i got this from regex101.com. and tested in search field in splunk. it was ok. unless it works differently?
yep. here are two sample logs
2014-06-04 23:42:42,115,,,1401889361349,MetricLogger,TDI_CLOUDCSX_1,1401889361349,hymlxsdfbpe11_1401889362113_11537,,RetrieveIdentityDetails,148
2014-06-04
23:42:36,427,,,0dedf85a-fbdb-43cb-b9f1-d4a0f636ab97,MetricLogger,TELSTRA_PREPAIDACTIVATION_STRATEGIC,0dedf85a-fbdb-43cb-b9f1-d4a0f636ab97,chslxsdfbpe05_1401889356427_2871,,CCandB.CreateNewBillingAccount,2983
i tried two methods,
FIRST method, just in props as below. does not quite work. worked when i use rex field=_raw "regex" though in search field though. tested in on one of those regex online as well
[sdf_bpel_metric]
EXTRACT-SDFCorepolicymetrics = (?:[^,\n],){5}(?P
SECOND method
in props
[sdf_bpel_metric]
REPORT-sdf_policy_metric = SDFCorepolicymetrics
in transforms
[SDFCorepolicymetrics]
FORMAT = requestApplicationLabel::$1 requestTransactionID::$2 callingApplication::$4 callType::$5 function::$6
REGEX = ([a-zA-Z]+),([^,]),([^,]),([^,]),([^,]),([^,]*),
And perhaps of the (relevant portions of) props.conf, and perhaps inputs.conf as well (only the portion where you configure the input of this file).
Can you provide a sample of the raw log?
This may indicate that the EXTRACT is not applied at all. Under what stanza header have you put the EXTRACT? Does this match the sourcetype/source/host?
Just want to point out that you don't need to reingest the log, or restart Splunk. Field extractions happen (mostly) at search-time, regardless of if they happen in props.conf/transforms.conf or inline in your search.
Yes, thats exactly what I have set and it doesnt work, no matter how much I perform a restart or Log Re-ingestion.
what's your props.conf settings? Is it the below or not?
EXTRACT-PHID= set=(?<phoneid>.+)\snotTime