Trouble with rex exceeding match limit

[splunkyhokie]

I'm trying to extract a field with the result of an API from a log, either containing "success" or "success.notfound".

This same API call is logged multiple times within a single event, so I'm trying to use rex to only look at the result listed immediately after a particular line item corresponding to the result I am interested in (I want the result after the "onePartKey=true" string, not the one after the "shippingstatus.within_call_digital_events_api_throttle.yes" string)

Using rex across multiple lines, with each line having a timestamp, makes it a little harder, I think, but I came up with a query I think should work.

It runs and gives me some partial data, but I get messages saying:

 "Errors occurred while the search was executing. Therefore, search results might be incomplete."  and "Streamed search execute failed because: Error in 'rex' command: regex="product\.rtm\.success, \/customers\/{custRefId}\/profiles\?onePartKey.true[\s\S]*product\.aws\.attempt, DigitalEventsAPI[\s\S]*product\.aws\.(?P(.*?)), \/private\/ASVEVENTHUB" has exceeded configured match_limit, consider raising the value in limits.conf."

Any ideas on how to fix this to get it to run without errors?

Here's my rex:

rex "product\.rtm\.success, \/customers\/{custRefId}\/profiles\?onePartKey.true[\s\S]*product\.aws\.attempt, DigitalEventsAPI[\s\S]*product\.aws\.(?P<Result>(.*?)), \/private\/ASVEVENTHUB"

The log looks like this:

* WANT THIS ONE *
189, 2019-03-31 20:58:35.670 PDT, product.rtm.success, /customers/{custRefId}/profiles?onePartKey=true

190, 2019-03-31 20:58:35.670 PDT, product.aws.attempt, DigitalEventsAPI
191, 2019-03-31 20:58:35.818 PDT, product.aws.success, /private/ASVEVENTHUB/servicing-events/{profileId}/?index=webUserId

...
* DON'T WANT THIS ONE *
244, 2019-03-31 20:58:37.820 PDT, product.shippingstatus.within_call_digital_events_api_throttle.yes,

245, 2019-03-31 20:58:37.971 PDT, product.aws.attempt, DigitalEventsAPI
246, 2019-03-31 20:58:38.119 PDT, product.aws.success.notfound, /private/ASVEVENTHUB/servicing-events/{profileId}/?index=webUserId

[rbechtold]

Try this regex to see if it helps:

| rex field=&ltyour_field_here&gt max_match=0 "onePartKey=true[\S\s]+?product.aws.(?&ltstatus&gtsuccess|success.notfound)\,"

This regex should be roughly 3x more efficient (440 steps down to 168 steps), and also uses many less literals (making it more flexible, as to not accidentally miss anything)

[\S\s]? should almost always be used in place of [\S\s] unless you're confident about how they differ. Without the question mark quantifier, the expression is incredibly greedy, and will continue to match everything until it can't possibly match anymore. This often leads unintentionally missing regex matches, and inefficiency on the back end .

To illustrate this concept, let's say we want to match all the "MATCH" words in this string:

______MATCH______MATCH______MATCH______

Using [\S\s]*(?&ltgreedy_expression&gtMATCH) as our expression, we only capture the third "MATCH". This is because the expression is greedy by nature, and wants to capture as much as possible. For that reason, the expression captures everything it can until the very last instance of the word "MATCH".

On the other hand, if we make the expression lazy by adding a question mark after the wildcard -- [\S\s]*?(?&ltlazy_expression&gtMATCH)

You'll notice all three "MATCH" words are captured AND it only took about half the steps as the greedy version.

I hope this helps and makes sense! You may still have problems with limits, depending on the actual size of each log, but this should at least make it much less resource intensive on the back end of your search!

[ragedsparrow]

This is from the limits.conf spec document for [rex]

match_limit = <integer>
* Limits the amount of resources that are spent by PCRE
  when running patterns that will not match.
* Use this to set an upper bound on how many times PCRE calls an internal
  function, match(). If set too low, PCRE might fail to correctly match
  a pattern.
* Default: 100000

What this error is telling you is that you are taking too many steps to match. You can try to reduce the number of steps used to match in an online editor like http://www.regex101.com. Try taking an one of your larger events and testing out your regex in there. Keep note of the steps above the regex input box.

You could also try this:
| rex max_match=0 ...

The last option I can think of is to adjust the match_limit setting in the limits.conf for the rex stanza to better accommodate the number of steps you need for appropriate matches.