Filtering windows security event logs with Regex

[andrewdidone]

Hi there.

We've been having issues with our DC's sending to much information across to Splunk and require assistance on creating some regex filtering strings, as we are not familiar with regex.

We are currently pulling windows security events from 2 Windows domain controllers and received issues with the amount events indexed which constantly violates or license.

We have windows logon events (event code: 4624) that capture both user information logons as well as machine logons. There are so many of these logon events that we dont need and would like to remove it in order to stay within the license limit.

The security events also have a large description included in the event under the event type "Message" that would like to be removed.

Here is an example of what we have:
Log Name: Security
Source: Microsoft-Windows-Security-Auditing
Date: 2/28/2014 10:25:51 AM
Event ID: 4624
Task Category: Logon
Level: Information
Keywords: Audit Success
User: N/A
Computer: Computer
Description:
An account was successfully logged on.

Subject:
Security ID: NULL SID
Account Name: -
Account Domain: -
Logon ID: 0x0

Logon Type: 3

New Logon:
Security ID: DOMAIN*USERNAME*
Account Name: username
Account Domain: DOMAIN
Logon ID: 0xb008f014
Logon GUID: {877a24e2-7fff-857b-30a6-e4f061536b11}

Process Information:
Process ID: 0x0
Process Name: -

Network Information:
Workstation Name:

Source Network Address: IP address
Source Port: 49914

Detailed Authentication Information:
Logon Process: Kerberos
Authentication Package: Kerberos
Transited Services: -
Package Name (NTLM only): -
Key Length: 0

This event is generated when a logon session is created. It is generated on the computer that was accessed.

The subject fields indicate the account on the local system which requested the logon. This is most commonly a service such as the Server service, or a local process such as Winlogon.exe or Services.exe.

The logon type field indicates the kind of logon that occurred. The most common types are 2 (interactive) and 3 (network).

The New Logon fields indicate the account for whom the new logon was created, i.e. the account that was logged on.

The network fields indicate where a remote logon request originated. Workstation name is not always available and may be left blank in some cases.

The authentication information fields provide detailed information about this specific logon request.
- Logon GUID is a unique identifier that can be used to correlate this event with a KDC event.
- Transited services indicate which intermediate services have participated in this logon request.
- Package name indicates which sub-protocol was used among the NTLM protocols.
- Key length indicates the length of the generated session key. This will be 0 if no session key was requested.

---

We recieve the same event for machine logons however it has the machine name with a $ in the name:

        Security ID:                   *DOMAIN*\*MACHINE*$
        Account Name:              *MACHINE*$

The request is pretty much this:

Create a regex for the props and transforms that will filter out ALL events that contain the "machine"$ and KEEP the events that contain a proper username. REMOVE the "Message" field from the events to reduce indexing size.

Any help will be greatly appreciated. Please let me know if it needs more clarification.

Thanks,
Andrew

[andrewdidone]

Thanks for that.
The small string returns events, but the "grow" string and initial search string does not.
Also, I should mention that the "Account Name" and "Seurity ID" are labeled as "Account_Name" and "Security_ID" in the Splunk fields. Perhaps that causes issues int the regex?

[lukejadamec]

I can test it in the morning.
Otherwise, the way you set it up looks good except - this does not target the Message field. You should be able to test the regex in a search string to fix it:
index=main EventCode=4624 | regex "(?msi)^EventCode=4624\D.*^Security\sID:\s+[^ ]\$.*^Account\s+Name:\s+[^ ]\$" |table EventCode
That should pull only 4624 events as specified, if you get nothing, then the regex needs fixing.
You can start small:
index=main EventCode=4624 | regex "(?msi)^EventCode=4624\D"
And grow:
index=main EventCode=4624 | regex "(?msi)^EventCode=4624\D.*^Security\sID:\s+[^ ]\$"

[SAF_IT]

Does this break when you render your events in XML format? That seems to have happened to me. Any thoughts on how to fix it, besides changing back?

[sys1pmp]

Can you explain how can I just filter out Security_ID="NULL SID". Need help to find exact regex

[andrewdidone]

Hi lukejadamec

Thanks for the quick reply. I tried this and sadly it didn't seem to work.

Breakdown of what I did:
-Created and placed props.conf and transforms.conf with config supplied above into the system/local folder of the Indexer.
-Restarted splunk via command line: "splunk restart".

I changed the time picker to 1 minute ago. Unfortunately I am still seeing the "Message" field as well as the events from the "machine"$ hots.
Would you have any further ideas on this. Anything will greatly appreciated.

Cheers,
Andrew

[lukejadamec]

What version of Splunk?