PCRE_ERROR_RECURSIONLIMIT for regex on transforms.conf

[fabiocaldas]

For a sourcetype nginx log error, I would like to index everything except data that contains : unlink text. Eg:

2015/01/28 13:26:44 [crit] 29614#0: unlink() "/media/ephemeral0/cache/vtex_files_nginx/0/d4/9d3194ccb8b1e93e6e041ca812155d40" failed (2: No such file or directory)

This's the kind of message I don't want to index

So I created a props.conf

[source::/var/log/nginx/error.log]
TRANSFORMS-setJanusEdgeData = sendToPurge,sendToIndexing

and also a transforms.conf

[sendToPurge]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[sendToIndexing]
REGEX = ^((?!: unlink).)*$
DEST_KEY = queue
FORMAT = indexQueue

It's works fine leaving only messages without ": unlink" text to be indexed. Despite the success if I look at my indexing peers the splunkd.log file it's there messages like that one:

ERROR Regex - Failed in pcre_exec: Error PCRE_ERROR_RECURSIONLIMIT for regex: ^((?!: unlink).)*$

Where is the problem with regex? Or where I can change the RECURSIONLIMIT?

[fabiocaldas]

Thanks richgallowaY for your help. At end the solution was much simplier I only use a regex to purge what I really don't want

That's was my solution after all:

props.conf

[source::/var/log/nginx/error.log]
TRANSFORMS-setJanusEdgeData = sendVarLogNginxErrorToPurge

transforms.conf

[sendVarLogNginxErrorToPurge]
REGEX = : unlink
DEST_KEY = queue
FORMAT = nullQueue

[richgalloway · Accepted solution]

Your regex string was expecting multiple instances of ": unlink" from the beginning to the end of the line - nothing else. This regex string matches the example you gave:

(?!: unlink)

I think you should swap your transforms. Have the events that contain ": unlink" go to nullQueue and the rest be indexed.

[fabiocaldas]

I tried the regex that you gave above ** (?!: unlink)** on https://regex101.com/#python using the sample that I put on question but it give a match always. It's strange

[richgalloway]

Don't use python-style regex strings. Use PCRE. I had good results with https://www.regex101.com/.

[fabiocaldas]

I tried a transforms like

[sendToPurge]
 REGEX = : unlink
 DEST_KEY = queue
 FORMAT = nullQueue

 [sendToIndexing]
 REGEX = .
 DEST_KEY = queue
 FORMAT = indexQueue

But i leave everything (including messages with unlink) to be indexed. Any sugestion?

[richgalloway]

Perhaps this?

[sendToPurge]
REGEX = (: unlink)
DEST_KEY = queue
FORMAT = nullQueue

[fabiocaldas]

That's was my solution after all. Purge only what I don't want:

props.conf
[source::/var/log/nginx/error.log]
TRANSFORMS-setJanusEdgeData = sendVarLogNginxErrorToPurge

transforms.conf
[sendVarLogNginxErrorToPurge]
REGEX = : unlink
DEST_KEY = queue
FORMAT = nullQueue

[fabiocaldas]

You mean use only one transformation to discard? I think it doesn't work, but I will be testing it

[richgalloway]

Sorry, I meant to show changes to the one stanza. The other one stays the same.

[richgalloway]

Please share some sample data that you expect the regex to match.

[fabiocaldas]

Hi richgalloway, I just edited the question with the sample data that I don't wan't to index