Having trouble in applying regex to blacklist Windows events on a Universal Forwarder.

[AL3Z]

Hi,

Looking for some assistance with Regex to blacklist  inputs.conf on Windows Systems.  We modified inputs.conf located:
/opt/apps/splunk/etc/deployment-apps/Splunk_TA_windows/local/inputs.conf

Applied Regex :

blacklist1 = EventCode="4688" $XmlRegex="<Data Name='NewProcessName'>
(C:\\Program Files\\Windows Defender Advanced Threat Protection\\MsSense.exe)|(C:\\Program Files (x86)\\Tanium\\Tanium Client\\TaniumCX.exe)
</Data>"

I attempted all available methods to blacklist the events above, but they did not take effect. Do we need to make modifications in order to successfully blacklist them?

Thanks

[gcusello]

Hi @AL3Z,

please try this regex:

\<EventID\>4688\<\/EventID\>.*\<Data Name\=\'NewProcessName\'\>.*(C:\\Program Files\\Windows Defender Advanced Threat Protection\\MsSense\.exe)|(C:\\Program Files \(x86\)\\Tanium\\Tanium Client\\TaniumCX\.exe)

that you can test at https://regex101.com/r/053rNX/1

Ciao.

Giuseppe

[AL3Z]

@gcusello 
Hi @gcusello/@richgalloway ,
This regex is not getting applied forthe events. I believe we need to  blacklist by using parent field ??

blacklist3 = $XmlRegex="<EventID\>4688\<\/EventID\>.*\<Data Name\=\'NewProcessName\'\>.*(C:\\Program Files\\Windows Defender Advanced Threat Protection\\MsSense\.exe)"

This is the actual log from EventViewer :

A new process has been created.

Creator Subject:

Security ID: SYSTEM

Account Name: SECUREJUMP$

Account Domain: EC

Logon ID: 0x3E7

Target Subject:

Security ID: NULL SID

Account Name: -

Account Domain: -

Logon ID: 0x0

Process Information:

New Process ID: 0x561c

New Process Name: C:\Program Files\Windows Defender Advanced Threat Protection\SenseCncProxy.exe

Token Elevation Type: %%1936

Mandatory Label: Mandatory Label\System Mandatory Level

Creator Process ID: 0x3520

Creator Process Name: C:\Program Files\Windows Defender Advanced Threat Protection\MsSense.exe

Process Command Line:

Thanks

[richgalloway]

I see a few problems here.

1. The blacklist1 setting is not in the proper format.  It must be a list of event IDs or a keyword followed by "=" followed by a regular expression.

2. The regex shown is trying to match XML, but the sample event is not in XML.

3. The regex is looking for text ("4688", "MsSense.exe", "TaniumCX.exe") that is not in the sample event.

Any of these would cause the blacklist to fail.  To fix them:

1. Put the blacklist1 setting in an expected format.

2. Examine the log entry as Splunk sees it (_raw) rather than as shown by another program (which may have changed it for display purposes).

3. Ensure the regex matches the sample data.

[AL3Z]

Hi @richgalloway ,

Need a clarification on blacklisting the field which one we need to put under blacklist is it newprocessname or parentprocessname ??

Thanks

[richgalloway]

The blacklist setting supports neither of those.  See my earlier reply for the list of supported keywords/fields.

[AL3Z]

Hi @richgalloway @gcusello ,

Is there any option where we can see the errors for the blacklisted regex  if it's not getting applied?

Thanks..

[richgalloway]

I'm not aware of any such option.  Perhaps one of the DEBUG log settings will help.

Failure to apply a regex is not an error - it just means the data doesn't match the regex, which is perfectly normal.

[AL3Z]

@richgalloway 

Where exactly we can see this debug log setting in the DS?

[richgalloway]

First, to answer the question posed in the OP, yes, you need to make modifications to successfully blacklist the events.  The regular expression must be valid and correct or it will not match the data and events will not be dropped as desired.

For instance, the '/' character must be escaped.  Literal parentheses (as in "Program Files(x86)") must be escaped.  There should not be any newlines in the expression.  Test the regex with matching and non-matching sample data at regex101.com.

Finally, I'm not positive about the debug log setting since I don't know that Splunk will log the information you seek.  If it does, however, it will be in the UF and not in the DS.  Go to Settings->Server settings->Server Logging and search for channels with "regex" in their names.  Set the value for likely candidates to DEBUG.  Be aware that this may be extremely verbose and should not be enabled for long.

[AL3Z]

@richgalloway 

Can you pls paste here the valid regex  for the above Event if possible.

Thanks..

[richgalloway]

This regex works with one of the two sample events.

<Data Name='NewProcessName'>(C:\\Program Files\\Windows Defender Advanced Threat Protection\\MsSense\.exe)|(C:\\Program Files \(x86\)\\Tanium\\Tanium Client\\TaniumCX\.exe)<\/Data>

[AL3Z]

Hi@richgalloway ,

Why there is no EventCode 4688 in the regex  ?
This is not working ,
https://regex101.com/r/45I3Kt/1 pls check it once 
Thanks

[richgalloway]

The EventCode key and $XmlRegex key use two different regular expressions.  The former is simple and certain to work correctly, whereas the latter is not.  That is why I showed a corrected $XmlRegex expression.

The regex101.com expression is working fine.  Include sample data that matches the expression and you'll see.

https://regex101.com/r/ZTE3Z4/1

[AL3Z]

Hi @richgalloway @gcusello ,

Despite testing multiple tests,  unable to achieve a blacklisting. Please, for the sake of accuracy address this issue.

blacklist5 = <Data Name='NewProcessName'>(C:\\Program Files\\Windows Defender Advanced Threat Protection\\SenseCncProxy\.exe)|(C:\\Program Files\\Windows Defender Advanced Threat Protection\\SenseCM\.exe)|(C:\\Program Files\\Windows Defender Advanced Threat Protection\\MsSense\.exe)|(C:\\Program Files\\WindowsPowerShell\\Modules\\gytpol\\Client\\fw.\_.\_.+\\GytpolClientFW.\_.\_.\.exe)|(C:\\Program Files\\Windows Defender Advanced Threat Protection\\SenseIR\.exe)|(C:\\Program Files\\Rapid7\\Insight Agent\\components\\insight_agent\\.*\\get_proxy.exe)|(C:\\Program Files\\Rapid7\\Insight Agent\\components\\insight_agent\\.*\\ir_agent\.exe)|(C:\\Program Files\\Windows Defender Advanced Threat Protection\\SenseCncProxy\.exe)|(C:\\Program Files\\Windows Defender Advanced Threat Protection\\MsSense\.exe)|C:\\Program Files\\Rapid7\\Insight Agent\\components\\insight_agent\\.*\\get_proxy.exe|(C:\\Program Files\\Rapid7\\Insight Agent\\components\\insight_agent\\.*\\ir_agent\.exe)|(C:\\Program Files \(x86\)\\Tanium\\Tanium Client\\TaniumCX\.exe)|(C:\\Program Files\\Windows Defender Advanced Threat Protection\\SenseCncProxy\.exe)|(C:\\Program Files\\Microsoft Monitoring Agent\\Agent\\MonitoringHost\.exe)|(C:\\Program Files\\Windows Defender Advanced Threat Protection\\MsSense\.exe)|(C:\\Program Files \(x86\)\\Tanium\\Tanium Client\\TaniumCX\.exe)|(C:\\Program Files\\AzureConnectedMachineAgent\\GCArcService\\GC\\gc_worker\.exe)|(C:\\Program Files\\SplunkUniversalForwarder\\bin\\splunk\.exe)<\/Data>

All Events matched with regex 

https://regex101.com/r/Xqw7eP/1

Thanks a lot..

[richgalloway]

The regex looks fine, although it's not necessary to escape underscore (_) characters.

The blacklist5 setting is missing "$XmlRegex=" and delimiters around the regex.

[AL3Z]

Hi @richgalloway ,

Thanks, 
How can we verify whether the logs are ingesting or not ? We've deployed the configuration to approximately 3,000 clients. Is there a way to check them all simultaneously?

[richgalloway]

The Deployment Server knows if the app containing the settings has been downloaded by each client.  To to Settings->Forwarder management and switch to the Apps tab.

[AL3Z]

I mean how we can query and confirm on search head like index=foo  parentprocessname="c:\\program file\\......"
to check the blacklisted events.

Thanks

[richgalloway]

Blacklisted events are not logged nor is there a log message when an event is blacklisted.  Therefore, there is nothing to search.  If the event exists on your Windows server and doesn't exist in Splunk then the blacklisting is successful.

[AL3Z]

• My question is how do we confirm it in Splunk?