Splunk Search

PCRE_ERROR_RECURSIONLIMIT for regex on transforms.conf

fabiocaldas
Contributor

For a sourcetype nginx log error, I would like to index everything except data that contains : unlink text. Eg:

2015/01/28 13:26:44 [crit] 29614#0: unlink() "/media/ephemeral0/cache/vtex_files_nginx/0/d4/9d3194ccb8b1e93e6e041ca812155d40" failed (2: No such file or directory)

This's the kind of message I don't want to index

So I created a props.conf

[source::/var/log/nginx/error.log]
TRANSFORMS-setJanusEdgeData = sendToPurge,sendToIndexing

and also a transforms.conf

[sendToPurge]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[sendToIndexing]
REGEX = ^((?!: unlink).)*$
DEST_KEY = queue
FORMAT = indexQueue

It's works fine leaving only messages without ": unlink" text to be indexed. Despite the success if I look at my indexing peers the splunkd.log file it's there messages like that one:

ERROR Regex - Failed in pcre_exec: Error PCRE_ERROR_RECURSIONLIMIT for regex: ^((?!: unlink).)*$

Where is the problem with regex? Or where I can change the RECURSIONLIMIT?

1 Solution

richgalloway
SplunkTrust
SplunkTrust

Your regex string was expecting multiple instances of ": unlink" from the beginning to the end of the line - nothing else. This regex string matches the example you gave:

(?!: unlink)

I think you should swap your transforms. Have the events that contain ": unlink" go to nullQueue and the rest be indexed.

---
If this reply helps you, Karma would be appreciated.

View solution in original post

fabiocaldas
Contributor

Thanks richgallowaY for your help. At end the solution was much simplier I only use a regex to purge what I really don't want

That's was my solution after all:

props.conf

[source::/var/log/nginx/error.log]
TRANSFORMS-setJanusEdgeData = sendVarLogNginxErrorToPurge

transforms.conf

[sendVarLogNginxErrorToPurge]
REGEX = : unlink
DEST_KEY = queue
FORMAT = nullQueue
0 Karma

richgalloway
SplunkTrust
SplunkTrust

Your regex string was expecting multiple instances of ": unlink" from the beginning to the end of the line - nothing else. This regex string matches the example you gave:

(?!: unlink)

I think you should swap your transforms. Have the events that contain ": unlink" go to nullQueue and the rest be indexed.

---
If this reply helps you, Karma would be appreciated.

fabiocaldas
Contributor

I tried the regex that you gave above ** (?!: unlink)** on https://regex101.com/#python using the sample that I put on question but it give a match always. It's strange

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Don't use python-style regex strings. Use PCRE. I had good results with https://www.regex101.com/.

---
If this reply helps you, Karma would be appreciated.
0 Karma

fabiocaldas
Contributor

I tried a transforms like

[sendToPurge]
 REGEX = : unlink
 DEST_KEY = queue
 FORMAT = nullQueue

 [sendToIndexing]
 REGEX = .
 DEST_KEY = queue
 FORMAT = indexQueue

But i leave everything (including messages with unlink) to be indexed. Any sugestion?

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Perhaps this?

[sendToPurge]
REGEX = (: unlink)
DEST_KEY = queue
FORMAT = nullQueue
---
If this reply helps you, Karma would be appreciated.

fabiocaldas
Contributor

That's was my solution after all. Purge only what I don't want:

props.conf
[source::/var/log/nginx/error.log]
TRANSFORMS-setJanusEdgeData = sendVarLogNginxErrorToPurge

transforms.conf
[sendVarLogNginxErrorToPurge]
REGEX = : unlink
DEST_KEY = queue
FORMAT = nullQueue

fabiocaldas
Contributor

You mean use only one transformation to discard? I think it doesn't work, but I will be testing it

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Sorry, I meant to show changes to the one stanza. The other one stays the same.

---
If this reply helps you, Karma would be appreciated.
0 Karma

richgalloway
SplunkTrust
SplunkTrust

Please share some sample data that you expect the regex to match.

---
If this reply helps you, Karma would be appreciated.
0 Karma

fabiocaldas
Contributor

Hi richgalloway, I just edited the question with the sample data that I don't wan't to index

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...