Splunk Search

Receiving message "Field extractor name=extract-doublecolon-transform is unusually slow" How to optimize the regex for my field extraction?

Venkat_16
Contributor

We have events in below format..

[2014-11-17 05:00:00,876] [INFO] [EventTimestamp::2014-11-17T05:00:00.876-06:00|ReferenceID::SomeID|ServiceName::Some.Services|OperationName::<null>|Direction::REQUEST|Server.Port::prod_domain.server1:1001|<xml>...some_big_xml_here...</xml>]

We applied below props/transforms to extract fields, with field_name on left side of :: and right side the value
(something similar to what splunk does by default with = sign in logs)

[extract-doublecolon-transform]
REGEX=([^\s\:]+)\:\:([^\|]+)\|
FORMAT=$1::$2

This regex works fine, however at times I receive below message.
Field extractor name=extract-doublecolon-transform is unusually slow
How do I best optimize the above regex for the sample event given above.

0 Karma

MuS
Legend

Hi Venkat_16,

The solutions are :
- improve the regexes/field extractions ( like this ([^\|\[]+)\:\:([^\|]+) ? )
- or change the warning threshold for key values extraction

edit $SPLUNK_HOME/etc/system/local/limits.conf, and change max_extractor_time value
see http://docs.splunk.com/Documentation/Splunk/6.2.0/Admin/Limitsconf for more details

[kv]
max_extractor_time = <integer>
* Maximum amount of CPU time, in milliseconds, that a key-value pair extractor will be allowed to 
* take before warning. If the extractor exceeds this execution time on any event a warning will be issued
* Defaults to 1000

avg_extractor_time = <integer>
* Maximum amount of CPU time, in milliseconds, that the average (over search results) execution time of 
* a key-value pair extractor will be allowed to take before warning. Once the average becomes larger 
* than this amount of time a warning will be issued
* Defaults to 500

hope this helps to sort things ...

cheers, MuS

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...