Splunk Search

How to blacklist events in WinEventLog that have a message containing quotes?

dstaulcu
Builder

I am playing with a new windows event log source called sysmon. Among other things, sysmon logs process creation events. In observation of process creation events, Splunk is by far the most noisy process on my test system. In trying to master advanced filtering techniques for wineventlogs, I figured I'd set about trying to blacklist any process create events supporting a splunk agent restart. I am having trouble getting a match on blacklist item #2 having quotes in match data. I've tried leaving the quotes unescaped and I have tried escaping the quotes using the backslash... Should this work? Is there another way?

Inputs appreciated:

[WinEventLog://Microsoft-Windows-Sysmon/Operational]
disabled = false
renderXml = false
# ignore process creates for process called by splunk.exe, splunkd.exe (nuclear option)
blacklist1=EventCode="1" Message="ParentImage: (C:\\Program Files\\SplunkUniversalForwarder\\bin\\splunkd.exe|C:\\Program Files\\SplunkUniversalForwarder\\bin\\splunk.exe)"
# ignore process creates for processes called by splunkd.exe restart task:   (restart scenario only)
blacklist2=EventCode="1" Message="ParentCommandLine: (\"C:\\Program Files\\SplunkUniversalForwarder\\bin\\splunkd.exe\" service)"
0 Karma

jplumsdaine22
Influencer

Are you using a Universal Forwarder?; I don't think you can filter data with it. See http://docs.splunk.com/Documentation/Splunk/6.3.2/Forwarding/Routeandfilterdatad

0 Karma

adambra
New Member

Same question: I can't filter with a particular word in Message.

I wanted to filter the messages containing "Source Network Address:192.168.0.1" witch is the supervision server.

I tried (with and without backslashes):
blacklist = Message="/^.*192\.168\.0\.1.*$/m"
blacklist = Message="^.*192\.168\.0\.1.*$"
blacklist = Message="*192\.168\.0\.1*"
blacklist = Message="192\.168\.0\.1"

I tried a lot of expressions without success.

Any suggestion about that?

0 Karma

dstaulcu
Builder

I've been doing a lot of testing today and have an update on the scope of this problem.

Filter attempts (whitelist or blacklist) on EventCode and Message key value data appear to behave differently when renderXml = True compared to when renderXml = False.

Taking the following Event Message data for example:

fragment_begin<Data Name='Image'>C:\Windows\System32\ping.exe</Data><Data Name='CommandLine'>fragment_end

When renderXml = False, the following expression succeeds in filtering events:

blacklist = Message="<Data Name='Image'>.*\\(calc|ping).exe</Data><Data Name='CommandLine'>" 

However, when renderXml = True, the same expression fails to filter events.

After trying a various filtering strategies on this Message data when renderXml = True, it appears that matching fails any time when the < and > and , and = chars are included in the text to match. I've tried cancelling these characters with a backslash (or two or three) but no amount of that seems to work. I imagine there are other impacted characters as well.

Here are some of the testing patterns I have tried:

#Working
#blacklist = EventCode="1" Message="notepad.exe"
#blacklist = EventCode="1" Message="CommandLine"   
#blacklist = EventCode="1" Message=":"
#blacklist = EventCode="1" Message="\\"
#blacklist = EventCode="1" Message="(notepad.exe|calc.exe)"   
#blacklist = EventCode="1" Message="(\\notepad.exe|calc.exe)"   
#blacklist = EventCode="1" Message="(?i)(\\notePad.exe|calc.exe)" 
#blacklist = EventCode="1" Message="(?i)\\(ping.exe|calc.exe)" 
#blacklist = EventCode="1" Message="(?i)\\(pIng|caLc).exe" 
#blacklist = EventCode="1" Message="(?i)\\(pIng|caLc).exe.*"

#Not working
#blacklist = EventCode="1" Message="(?i)\\(pIng|caLc).exe\<"
#blacklist = EventCode="1" Message="(?i)\\(pIng|caLc).exe\<"
#blacklist = EventCode="1" Message="(?i)\\(pIng|caLc).exe\\<"
#blacklist = EventCode="1" Message="(?i)\\(pIng|caLc).exe\\\<"
#blacklist = EventCode="1" Message="(?i)\\(pIng|caLc).exe\\\\<"
#blacklist = EventCode="1" Message="'CommandLine'"
#blacklist = EventCode="1" Message="'='"  
#blacklist = EventCode="1" Message="'\='"
#blacklist = EventCode="1" Message="\."
0 Karma

nnmiller
Contributor

Please try using one of the following to work around this issue:

Unicode or ASCII version of special characters such as backslash, "<", or, ">". For example:
Unicode "\" = \u005c
Hex ASCII "\" = \x5c
Unicode "<" = u003c
Hex ASCII "<" = \x3c

Use {X} where "X" is the character count to indicate the number of occurrences of the character. E.g.:

#blacklist = EventCode="1" Message="(?i)\\(pIng|caLc).exe\<{1}"
#blacklist = EventCode="1" Message="\.{1}"

In SEDCMD these two methods are used as a workaround for oddities with the regex parser, so I am wondering if they work here as well.

0 Karma

dstaulcu
Builder

Thanks for the suggestion. Have to revisit this problem for another event type in today. Will give your suggestion a shot.

Meanwhile, here are some fill-in-the-blank blacklist patterns for all event types from Sysmon:

    # Sample blacklist entries for Sysmon schemaversion 4.21
blacklist1 = EventCode="^255$" Message="(?i)^UtcTime:\s+(.*)\s+ID:\s+(.*)\s+Description:\s+(.*)\s+Description:\s+(.*)$"

blacklist2 = EventCode="^1$" Message="(?i)^RuleName:\s+(.*)\s+UtcTime:\s+(.*)\s+ProcessGuid:\s+(.*)\s+ProcessId:\s+(.*)\s+Image:\s+(.*)\s+FileVersion:\s+(.*)\s+Description:\s+(.*)\s+Product:\s+(.*)\s+Company:\s+(.*)\s+OriginalFileName:\s+(.*)\s+CommandLine:\s+(.*)\s+CurrentDirectory:\s+(.*)\s+User:\s+(.*)\s+LogonGuid:\s+(.*)\s+LogonId:\s+(.*)\s+TerminalSessionId:\s+(.*)\s+IntegrityLevel:\s+(.*)\s+Hashes:\s+(.*)\s+ParentProcessGuid:\s+(.*)\s+ParentProcessId:\s+(.*)\s+ParentImage:\s+(.*)\s+ParentCommandLine:\s+(.*)\s+ParentCommandLine:\s+(.*)$"

blacklist3 = EventCode="^2$" Message="(?i)^RuleName:\s+(.*)\s+UtcTime:\s+(.*)\s+ProcessGuid:\s+(.*)\s+ProcessId:\s+(.*)\s+Image:\s+(.*)\s+TargetFilename:\s+(.*)\s+CreationUtcTime:\s+(.*)\s+PreviousCreationUtcTime:\s+(.*)\s+PreviousCreationUtcTime:\s+(.*)$"

blacklist4 = EventCode="^3$" Message="(?i)^RuleName:\s+(.*)\s+UtcTime:\s+(.*)\s+ProcessGuid:\s+(.*)\s+ProcessId:\s+(.*)\s+Image:\s+(.*)\s+User:\s+(.*)\s+Protocol:\s+(.*)\s+Initiated:\s+(.*)\s+SourceIsIpv6:\s+(.*)\s+SourceIp:\s+(.*)\s+SourceHostname:\s+(.*)\s+SourcePort:\s+(.*)\s+SourcePortName:\s+(.*)\s+DestinationIsIpv6:\s+(.*)\s+DestinationIp:\s+(.*)\s+DestinationHostname:\s+(.*)\s+DestinationPort:\s+(.*)\s+DestinationPortName:\s+(.*)\s+DestinationPortName:\s+(.*)$"

blacklist5 = EventCode="^4$" Message="(?i)^UtcTime:\s+(.*)\s+State:\s+(.*)\s+Version:\s+(.*)\s+SchemaVersion:\s+(.*)\s+SchemaVersion:\s+(.*)$"

blacklist6 = EventCode="^5$" Message="(?i)^RuleName:\s+(.*)\s+UtcTime:\s+(.*)\s+ProcessGuid:\s+(.*)\s+ProcessId:\s+(.*)\s+Image:\s+(.*)\s+Image:\s+(.*)$"

blacklist7 = EventCode="^6$" Message="(?i)^RuleName:\s+(.*)\s+UtcTime:\s+(.*)\s+ImageLoaded:\s+(.*)\s+Hashes:\s+(.*)\s+Signed:\s+(.*)\s+Signature:\s+(.*)\s+SignatureStatus:\s+(.*)\s+SignatureStatus:\s+(.*)$"

blacklist8 = EventCode="^7$" Message="(?i)^RuleName:\s+(.*)\s+UtcTime:\s+(.*)\s+ProcessGuid:\s+(.*)\s+ProcessId:\s+(.*)\s+Image:\s+(.*)\s+ImageLoaded:\s+(.*)\s+FileVersion:\s+(.*)\s+Description:\s+(.*)\s+Product:\s+(.*)\s+Company:\s+(.*)\s+OriginalFileName:\s+(.*)\s+Hashes:\s+(.*)\s+Signed:\s+(.*)\s+Signature:\s+(.*)\s+SignatureStatus:\s+(.*)\s+SignatureStatus:\s+(.*)$"

blacklist9 = EventCode="^8$" Message="(?i)^RuleName:\s+(.*)\s+UtcTime:\s+(.*)\s+SourceProcessGuid:\s+(.*)\s+SourceProcessId:\s+(.*)\s+SourceImage:\s+(.*)\s+TargetProcessGuid:\s+(.*)\s+TargetProcessId:\s+(.*)\s+TargetImage:\s+(.*)\s+NewThreadId:\s+(.*)\s+StartAddress:\s+(.*)\s+StartModule:\s+(.*)\s+StartFunction:\s+(.*)\s+StartFunction:\s+(.*)$"

blacklist10 = EventCode="^9$" Message="(?i)^RuleName:\s+(.*)\s+UtcTime:\s+(.*)\s+ProcessGuid:\s+(.*)\s+ProcessId:\s+(.*)\s+Image:\s+(.*)\s+Device:\s+(.*)\s+Device:\s+(.*)$"

blacklist11 = EventCode="^10$" Message="(?i)^RuleName:\s+(.*)\s+UtcTime:\s+(.*)\s+SourceProcessGUID:\s+(.*)\s+SourceProcessId:\s+(.*)\s+SourceThreadId:\s+(.*)\s+SourceImage:\s+(.*)\s+TargetProcessGUID:\s+(.*)\s+TargetProcessId:\s+(.*)\s+TargetImage:\s+(.*)\s+GrantedAccess:\s+(.*)\s+CallTrace:\s+(.*)\s+CallTrace:\s+(.*)$"

blacklist12 = EventCode="^11$" Message="(?i)^RuleName:\s+(.*)\s+UtcTime:\s+(.*)\s+ProcessGuid:\s+(.*)\s+ProcessId:\s+(.*)\s+Image:\s+(.*)\s+TargetFilename:\s+(.*)\s+CreationUtcTime:\s+(.*)\s+CreationUtcTime:\s+(.*)$"

blacklist13 = EventCode="^12$" Message="(?i)^RuleName:\s+(.*)\s+EventType:\s+(.*)\s+UtcTime:\s+(.*)\s+ProcessGuid:\s+(.*)\s+ProcessId:\s+(.*)\s+Image:\s+(.*)\s+TargetObject:\s+(.*)\s+TargetObject:\s+(.*)$"

blacklist14 = EventCode="^13$" Message="(?i)^RuleName:\s+(.*)\s+EventType:\s+(.*)\s+UtcTime:\s+(.*)\s+ProcessGuid:\s+(.*)\s+ProcessId:\s+(.*)\s+Image:\s+(.*)\s+TargetObject:\s+(.*)\s+Details:\s+(.*)\s+Details:\s+(.*)$"

blacklist15 = EventCode="^14$" Message="(?i)^RuleName:\s+(.*)\s+EventType:\s+(.*)\s+UtcTime:\s+(.*)\s+ProcessGuid:\s+(.*)\s+ProcessId:\s+(.*)\s+Image:\s+(.*)\s+TargetObject:\s+(.*)\s+NewName:\s+(.*)\s+NewName:\s+(.*)$"

blacklist16 = EventCode="^15$" Message="(?i)^RuleName:\s+(.*)\s+UtcTime:\s+(.*)\s+ProcessGuid:\s+(.*)\s+ProcessId:\s+(.*)\s+Image:\s+(.*)\s+TargetFilename:\s+(.*)\s+CreationUtcTime:\s+(.*)\s+Hash:\s+(.*)\s+Hash:\s+(.*)$"

blacklist17 = EventCode="^16$" Message="(?i)^UtcTime:\s+(.*)\s+Configuration:\s+(.*)\s+ConfigurationFileHash:\s+(.*)\s+ConfigurationFileHash:\s+(.*)$"

blacklist18 = EventCode="^17$" Message="(?i)^RuleName:\s+(.*)\s+EventType:\s+(.*)\s+UtcTime:\s+(.*)\s+ProcessGuid:\s+(.*)\s+ProcessId:\s+(.*)\s+PipeName:\s+(.*)\s+Image:\s+(.*)\s+Image:\s+(.*)$"

blacklist19 = EventCode="^18$" Message="(?i)^RuleName:\s+(.*)\s+EventType:\s+(.*)\s+UtcTime:\s+(.*)\s+ProcessGuid:\s+(.*)\s+ProcessId:\s+(.*)\s+PipeName:\s+(.*)\s+Image:\s+(.*)\s+Image:\s+(.*)$"

blacklist20 = EventCode="^19$" Message="(?i)^RuleName:\s+(.*)\s+EventType:\s+(.*)\s+UtcTime:\s+(.*)\s+Operation:\s+(.*)\s+User:\s+(.*)\s+EventNamespace:\s+(.*)\s+Name:\s+(.*)\s+Query:\s+(.*)\s+Query:\s+(.*)$"

blacklist21 = EventCode="^20$" Message="(?i)^RuleName:\s+(.*)\s+EventType:\s+(.*)\s+UtcTime:\s+(.*)\s+Operation:\s+(.*)\s+User:\s+(.*)\s+Name:\s+(.*)\s+Type:\s+(.*)\s+Destination:\s+(.*)\s+Destination:\s+(.*)$"

blacklist22 = EventCode="^21$" Message="(?i)^RuleName:\s+(.*)\s+EventType:\s+(.*)\s+UtcTime:\s+(.*)\s+Operation:\s+(.*)\s+User:\s+(.*)\s+Consumer:\s+(.*)\s+Filter:\s+(.*)\s+Filter:\s+(.*)$"

blacklist23 = EventCode="^22$" Message="(?i)^RuleName:\s+(.*)\s+UtcTime:\s+(.*)\s+ProcessGuid:\s+(.*)\s+ProcessId:\s+(.*)\s+QueryName:\s+(.*)\s+QueryStatus:\s+(.*)\s+QueryResults:\s+(.*)\s+Image:\s+(.*)\s+Image:\s+(.*)$"
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...