Splunk Search

Receiving message "Field extractor name=extract-doublecolon-transform is unusually slow" How to optimize the regex for my field extraction?

Venkat_16
Contributor

We have events in below format..

[2014-11-17 05:00:00,876] [INFO] [EventTimestamp::2014-11-17T05:00:00.876-06:00|ReferenceID::SomeID|ServiceName::Some.Services|OperationName::<null>|Direction::REQUEST|Server.Port::prod_domain.server1:1001|<xml>...some_big_xml_here...</xml>]

We applied below props/transforms to extract fields, with field_name on left side of :: and right side the value
(something similar to what splunk does by default with = sign in logs)

[extract-doublecolon-transform]
REGEX=([^\s\:]+)\:\:([^\|]+)\|
FORMAT=$1::$2

This regex works fine, however at times I receive below message.
Field extractor name=extract-doublecolon-transform is unusually slow
How do I best optimize the above regex for the sample event given above.

0 Karma

MuS
Legend

Hi Venkat_16,

The solutions are :
- improve the regexes/field extractions ( like this ([^\|\[]+)\:\:([^\|]+) ? )
- or change the warning threshold for key values extraction

edit $SPLUNK_HOME/etc/system/local/limits.conf, and change max_extractor_time value
see http://docs.splunk.com/Documentation/Splunk/6.2.0/Admin/Limitsconf for more details

[kv]
max_extractor_time = <integer>
* Maximum amount of CPU time, in milliseconds, that a key-value pair extractor will be allowed to 
* take before warning. If the extractor exceeds this execution time on any event a warning will be issued
* Defaults to 1000

avg_extractor_time = <integer>
* Maximum amount of CPU time, in milliseconds, that the average (over search results) execution time of 
* a key-value pair extractor will be allowed to take before warning. Once the average becomes larger 
* than this amount of time a warning will be issued
* Defaults to 500

hope this helps to sort things ...

cheers, MuS

0 Karma
Get Updates on the Splunk Community!

More Ways To Control Your Costs With Archived Metrics | Register for Tech Talk

Tuesday, May 14, 2024  |  11AM PT / 2PM ET Register to Attend Join us for this Tech Talk and learn how to ...

.conf24 | Personalize your .conf experience with Learning Paths!

Personalize your .conf24 Experience Learning paths allow you to level up your skill sets and dive deeper ...

Threat Hunting Unlocked: How to Uplevel Your Threat Hunting With the PEAK Framework ...

WATCH NOWAs AI starts tackling low level alerts, it's more critical than ever to uplevel your threat hunting ...