Splunk Search

inputs.conf - blacklist regex performance assistance

antb
Path Finder

Thank you in advance. Looking for some assistance with inputs.conf on Windows Systems. First, we modified inputs.conf located:
/opt/apps/splunk/etc/deployment-apps/Splunk_TA_windows/local/inputs.conf

1) Do we need to do anything for our splunk clients to pick up changes?
2) In terms of performance, and syntax does anyone have any concerns or recommendations to improve the performance on the 6 blacklists below? We believe they work but are unsure on performance.

[WinEventLog://Security]
disabled = 0
start_from = oldest
current_only = 0
evt_resolve_ad_obj = 1
checkpointInterval = 5
renderXml = false
index = windowsevents

blacklist1 = EventCode="4662" Message="(?i)Object Type:(?!\s*groupPolicyContainer)"
blacklist2 = EventCode="566" Message="(?i)Object Type:(?!\s*groupPolicyContainer)"
blacklist3 = 5154,4663,4689,5152,4627
blacklist4 = EventCode="4688" Message="(?im:New Process Name:).*(?i:SplunkUniversalForwarder\\bin\\)(?i:splunk\.exe|btool\.exe)"
blacklist5 = EventCode="4688" ComputerName="verybadscripts\.myco\.com" Message="(?im:New Process Name:.*(\\grep\.exe|\\awk\.exe))"
Blacklist6 = EventCode="(4624|4634|4672)" ComputerName="(?i:(.+noisycomp|.+loudercomp).+\.myco\.com)" Message="(?im:.*Account Name:\s+.*(noisycomp|loudercomp).*\$)"

As an example computers have names like abloudercomp01 and bcdloudercompx02 and so the account names would be abloudercomp01$ and bcdloudercompx02$ respectfully... I don't want to presume two chars followed by two numbers but can presume <=5 chars on either side of the loudercomp string(s) if it would make a large difference in performance.

For some reason, we aren't currently seeing any 4688 events at all (from any comp) after the BL 4 and 5 were added early today but it could just be that there isn't a lot of volume on the weekend.

0 Karma
1 Solution

moliminous
Path Finder

1) Assuming your Deployment Server (DS) is configured and the appropriate clients belong to Server Classes with the App, you just need to reload the DS by running the following command on the CLI of the DS:

/opt/splunk/bin/splunk reload deploy-server

For more information:
https://docs.splunk.com/Documentation/Splunk/7.3.2/Updating/Updateconfigurations#Redeploy_an_app_aft...

2) The first few regex statements should be the simpler ones that filter out the most events quickly, that way there are less events that the more complex regex has to process.
Specifically, I would move blacklist3 up to be blacklist1, and shift the rest down. Also, I would put blacklist5 above blacklist4 since it is more specific by having ComputerName without wildcards.

Without seeing the data it would be difficult to make other specific suggestions, but try these generic suggestions:

  • Limit the wildcards as much as possible
  • Put the raw logs in regex101.com (or similar site) and test your regex to ensure it matches all the desired events while being as specific as possible

Hope this helps!

View solution in original post

woodcock
Esteemed Legend

I usually just restart splunk on the DS (does not require auth) to make all changes active.

0 Karma

moliminous
Path Finder

1) Assuming your Deployment Server (DS) is configured and the appropriate clients belong to Server Classes with the App, you just need to reload the DS by running the following command on the CLI of the DS:

/opt/splunk/bin/splunk reload deploy-server

For more information:
https://docs.splunk.com/Documentation/Splunk/7.3.2/Updating/Updateconfigurations#Redeploy_an_app_aft...

2) The first few regex statements should be the simpler ones that filter out the most events quickly, that way there are less events that the more complex regex has to process.
Specifically, I would move blacklist3 up to be blacklist1, and shift the rest down. Also, I would put blacklist5 above blacklist4 since it is more specific by having ComputerName without wildcards.

Without seeing the data it would be difficult to make other specific suggestions, but try these generic suggestions:

  • Limit the wildcards as much as possible
  • Put the raw logs in regex101.com (or similar site) and test your regex to ensure it matches all the desired events while being as specific as possible

Hope this helps!

Get Updates on the Splunk Community!

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...

Splunk Custom Visualizations App End of Life

The Splunk Custom Visualizations apps End of Life for SimpleXML will reach end of support on Dec 21, 2024, ...