Splunk Search

Receiving message "Field extractor name=extract-doublecolon-transform is unusually slow" How to optimize the regex for my field extraction?

Venkat_16
Contributor

We have events in below format..

[2014-11-17 05:00:00,876] [INFO] [EventTimestamp::2014-11-17T05:00:00.876-06:00|ReferenceID::SomeID|ServiceName::Some.Services|OperationName::<null>|Direction::REQUEST|Server.Port::prod_domain.server1:1001|<xml>...some_big_xml_here...</xml>]

We applied below props/transforms to extract fields, with field_name on left side of :: and right side the value
(something similar to what splunk does by default with = sign in logs)

[extract-doublecolon-transform]
REGEX=([^\s\:]+)\:\:([^\|]+)\|
FORMAT=$1::$2

This regex works fine, however at times I receive below message.
Field extractor name=extract-doublecolon-transform is unusually slow
How do I best optimize the above regex for the sample event given above.

0 Karma

MuS
Legend

Hi Venkat_16,

The solutions are :
- improve the regexes/field extractions ( like this ([^\|\[]+)\:\:([^\|]+) ? )
- or change the warning threshold for key values extraction

edit $SPLUNK_HOME/etc/system/local/limits.conf, and change max_extractor_time value
see http://docs.splunk.com/Documentation/Splunk/6.2.0/Admin/Limitsconf for more details

[kv]
max_extractor_time = <integer>
* Maximum amount of CPU time, in milliseconds, that a key-value pair extractor will be allowed to 
* take before warning. If the extractor exceeds this execution time on any event a warning will be issued
* Defaults to 1000

avg_extractor_time = <integer>
* Maximum amount of CPU time, in milliseconds, that the average (over search results) execution time of 
* a key-value pair extractor will be allowed to take before warning. Once the average becomes larger 
* than this amount of time a warning will be issued
* Defaults to 500

hope this helps to sort things ...

cheers, MuS

0 Karma
Get Updates on the Splunk Community!

Observability | Use Synthetic Monitoring for Website Metadata Verification

If you are on Splunk Observability Cloud, you may already have Synthetic Monitoringin your observability ...

More Ways To Control Your Costs With Archived Metrics | Register for Tech Talk

Tuesday, May 14, 2024  |  11AM PT / 2PM ET Register to Attend Join us for this Tech Talk and learn how to ...

.conf24 | Personalize your .conf experience with Learning Paths!

Personalize your .conf24 Experience Learning paths allow you to level up your skill sets and dive deeper ...