Getting Data In

Why am I unable to reduce the amount of data indexed for [WinEventLog://Security] through blacklists or suppress_text options?

Jhand2016
Explorer

We are a new Splunk Enterprise 6.4.1 installation and have ran into a snag with indexing our Domain Controller logs. Due to the sheer volume and limited license of 50GB, we had to turn off the Universal Forwarder on 7 out of 8 DCs in our environment. However, we are still getting upwards of 20GB indexed from 1 DC each day. We have used blacklist successfully for the bulk of the noisy events. We also attempted to use the suppress_text=1 argument, but it does not actually strip any of the message or body within the events.

For our situation, we are indexing the [WinEventLog://Security] to capture user login/logoff details within the InfoSec realm. We found that 90% of our EventCodes are 4624 and 4634. These two events are actually the events we need as they capture login/logoff transactions, however, they include events within them for all types of transactions to include NTLM, Kerberos, token exchange, session closes, and machine account access. We only need the user logon/logoff related events. We used the below inputs.conf placed in the "Splunk_TA_windows\local\" Directory. Am I missing something? Can we use a regex to exclude certain types of vents from within the EventCode 4624.

[default]
host = DC-name

[WinEventLog:Security]
disabled = 0
suppress_text = 1

[WinEventLog://Security]
disabled=0
current_only=1
blacklist=EventCode=4656,4658,4670,4690,4663,5140

[script://$SPLUNK_HOME\bin\scripts\splunk-wmi.path]
disabled = 0

martynoconnor
Communicator

An entry in props.conf can also help reduce the amount of data ingested by Windows events without removing meaningful values:

[WinEventLog:Security]
SEDCMD-remwinstr = s/(?ism)(Token Elevation Type indicates|This event is generated).*$//g
0 Karma

spayneort
Contributor

You need to put the suppress_text = 1 under [WinEventLog://Security], not under [WinEventLog:Security].

You can use regex in whitelists and blacklists, see here:
http://docs.splunk.com/Documentation/Splunk/6.4.1/Data/MonitorWindowseventlogdata

0 Karma

Jhand2016
Explorer

Thanks, we tried moving the suppress text out of the 1st set and under the [WinEventLog://Security] and it actually had an interesting effect, it scrubbed the 6424/6434 messages completely out of the events. We still saw them on the DC event log, however they would not be indexed at all, it actually did this to just about any event type longer that about 20 lines. As soon as we comment out the suppress text they all populate and index again???

Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...