Hi All,
I am having difficulty finding in-depth documentation on REGEX syntax, and I am attempting to filter out [WinEventLog:Security] logs from our central Splunk Instance by use of a heavy forwarder.
On our Heavy Forwarder, we have the following filters setup:
props.conf
[WinEventLog:Security]
TRANSFORMS-security= npm-setnull, everythingElse
Filter NPM account Security logs to nullQueue
[npm-setnull]
REGEX = (?msi)Account_Name="ACCOUNT@DOMAIN.COM"
DEST_KEY = queue
FORMAT = nullQueue
Send everything else to be indexed in the indexQueue
[everythingElse]
REGEX = .
DEST_KEY = queue
FORMAT = indexQueue
This is currently not filtering any of the logs, and I feel thate the syntax is incorrect, but I can't find any solid documentation on how to format this. Can anyone help?
Thank you in advance,r
Daniel
You'll need to reverse the order in which the transforms are called from props.conf. Or in your case, just remove the [everythingElse] stanza and the reference to it. While it might seem a bit contra-intuitive, think of it like this;
"All events will pass through all transforms in the specified order before being returned for further processing"
It is not like there is some kind of 'break-on-match' processing. Thus, first setting the queue to nullQueue for some events, and then immediately resetting it back to indexQueue (for all events), will have the same results as not having any filtering at all.
UPDATE:
Yes, wolverine is right. The field name Account_Name
is a so-called 'cleaned' name, i.e, spaces are replaced with underscores etc. The REGEX
must match the actual text in the event, i.e.;
REGEX = Account\sName:\s+ACCOUNT@DOMAIN.COM
/k
/K
You'll need to reverse the order in which the transforms are called from props.conf. Or in your case, just remove the [everythingElse] stanza and the reference to it. While it might seem a bit contra-intuitive, think of it like this;
"All events will pass through all transforms in the specified order before being returned for further processing"
It is not like there is some kind of 'break-on-match' processing. Thus, first setting the queue to nullQueue for some events, and then immediately resetting it back to indexQueue (for all events), will have the same results as not having any filtering at all.
UPDATE:
Yes, wolverine is right. The field name Account_Name
is a so-called 'cleaned' name, i.e, spaces are replaced with underscores etc. The REGEX
must match the actual text in the event, i.e.;
REGEX = Account\sName:\s+ACCOUNT@DOMAIN.COM
/k
/K
Yes, you need to restart splunkd - either from the outside, like restarting the service, or from the inside, in the GUI, or hit the refresh endpoint url;
http(s)://your_splunk:8000/en-US/debug/refresh
That will also reload most configs.
Purely search-related configs, such as field extractions will be reloaded for each search.
/k
Thanks guys, I managed to get the filter working.
I changed the field to look for the Logon Account
[npm-setnull]
REGEX = Logon\sAccount:\snpm
DEST_KEY = queue
FORMAT = nullQueue
But I believe the Splunkd service in Services.msc needs to be rebooted in order to start the filter.
Regards!
Daniel
Awesome, thanks Kristian!
As it turns out, this did not fix the issue. I will have to dig deeper in this.
Thanks for your continued help, guys.
Daniel
Hi dscoland,
regexr.com
are two commonly referenced guides for learning/testing regular expressions.
Hi wolverine!
Thank you for your response. I am going to try this out. I know I probably sound like a broken record. But does Splunk provide REGEX documentation? I feel rather ignorant not understanding (?msi) and (?m) syntax; or is this provided elsewhere?
Thanks for your help, guys.
Daniel
good spotting, wolverine! Updated my answer to reflect on this as well.
/k
I think the issue might be with your REGEX:
REGEX = (?m)Account_Name="ACCOUNT@DOMAIN.COM"
Hi Kristian,
Thank you for your response.
I now have:
[WinEventLog:Security]
TRANSFORMS-security= npm-setnull
[npm-setnull]
REGEX = (?msi)Account_Name="NPM@EDM.LOCAL"
DEST_KEY = queue
FORMAT = nullQueue
It still appears not to be filtering. Do I need to restart the Heavy Forwarder instance?
Daniel