Problems blacklisting multiple eventcodes and blacklist items

[jh007]

I am attempting to update my input.confs list with the following blacklist:

blacklist1 = EventCode="4688|4648|4674" Message=".*[\S\s]*Account\sName:\s+[\S+]+[\$]" 
blacklist2 = EventCode="4624" Message=".*[\S\s]*Account\sName:\s+[\S+]+[SYSTEM]" 
blacklist3 = EventCode="4688" Message=".*[\S\s]*Account\sName:\s+[\S+]+[dragoncollect]"

Although the blacklists seem to be working mostly, the blacklist has unfortunately created a performance issue with capturing all 4688 windows events; in other words, now only a portion of my 4688 events are getting captured and reported in the Splunk server. I have generated several 4688 events on a local box (with a working forwarder), but none of the events I generate seem to show, yet other 4688 events are being captured in Splunk. I have verified that forwarders are installed on the boxes I am using and that all other eventcodes in the Splunk Sever are being captured.

Is there something wrong with my blacklist that could be causing the problem? could the problem be caused by me calling out the 4688 eventcode in two separate entries? Any help would be greatly appreciated.

Thanks!

---

marked code.

[DalJeanis · Accepted solution]

This is probably your performance killer -

.*[\S\s]*

The first item, .*, matches any number of anything. The second item [\S\s]*, matches any number of anything that is or isn't white space. (The only difference is that .* includes some word boundaries and other special characters. )

When the system encounters them, it basically has to remember everything it does, because it might have to back up and try again. In fact, it WILL have to back up and try again at each and every character for every relevant event that does NOT get blacklisted. So, for example, if the message is 20 characters long, then it will do about (20*19/2) = 190 steps, but if it is 40 characters long, it will do about (40*39/2)=780 steps, and if it is 80 characters long, it will do about (80*79/2)=1580 steps. In this case, you can just delete the [\S\s]* from all of those regexes and the performance will improve with no change to the output.

However, I would suggest that there are other issues with the regular expressions. This chunk of code [SYSTEM] means to match any single character in the list SYSTEM, which would be more effectively written as [EMSTY] If the intent is to match the word SYSTEM surrounded by square braces, then you need to escape the square braces, such as \[SYSTEM\].

Take some sample events and your regex over to regex101.com and validate that it is doing exactly what you want it to.

---

With regard to your 4688 problem, yes, you would be better off setting up a single blacklist that would process the 4688 records once and kill both types that you want to kill.