We have events in below format..
[2014-11-17 05:00:00,876] [INFO] [EventTimestamp::2014-11-17T05:00:00.876-06:00|ReferenceID::SomeID|ServiceName::Some.Services|OperationName::<null>|Direction::REQUEST|Server.Port::prod_domain.server1:1001|<xml>...some_big_xml_here...</xml>]
We applied below props/transforms to extract fields, with field_name on left side of :: and right side the value
(something similar to what splunk does by default with = sign in logs)
[extract-doublecolon-transform]
REGEX=([^\s\:]+)\:\:([^\|]+)\|
FORMAT=$1::$2
This regex works fine, however at times I receive below message.
Field extractor name=extract-doublecolon-transform is unusually slow
How do I best optimize the above regex for the sample event given above.
Hi Venkat_16,
The solutions are :
- improve the regexes/field extractions ( like this ([^\|\[]+)\:\:([^\|]+)
? )
- or change the warning threshold for key values extraction
edit $SPLUNK_HOME/etc/system/local/limits.conf, and change max_extractor_time value
see http://docs.splunk.com/Documentation/Splunk/6.2.0/Admin/Limitsconf for more details
[kv]
max_extractor_time = <integer>
* Maximum amount of CPU time, in milliseconds, that a key-value pair extractor will be allowed to
* take before warning. If the extractor exceeds this execution time on any event a warning will be issued
* Defaults to 1000
avg_extractor_time = <integer>
* Maximum amount of CPU time, in milliseconds, that the average (over search results) execution time of
* a key-value pair extractor will be allowed to take before warning. Once the average becomes larger
* than this amount of time a warning will be issued
* Defaults to 500
hope this helps to sort things ...
cheers, MuS