We're just now starting to enable some of the Windows Monitoring inputs to prepare for a deployment of the Splunk App for Windows Infrastructure. Not surprisingly, enabling the Network Monitor (WinNetMon stanzas) chewed right through our daily index license on a virtual server. That virtual server uses an offload server to handle AV scanning, so the top remoteAddress is to that offload scan server and the localhost address. What I'd like to do is filter those addresses out so we can still use the Network Monitoring, but we lose the stuff that we don't care about. I imagine we'll have the same issue on AD servers and we'll need to filter out other AD servers for replication purposes.
The remoteAddress portion of a WinNetMon configuration in inputs.conf seems to indicate that it accepts regular expressions. I'm not a RegEx expert by any means, so I'm looking for some help. What is the best format for a line that would filter out 2 or more remote addresses?
The following doesn't seem like it would do the job based on some regex testers out there:
remoteAddress = (!(192..168.0.1|127.0.0.1))
Thanks!
Unfortunately, regex doesn't support negation. You'll have to come up with a regex that specifies the addresses you want to see rather than those you do not want to see.
Alternatively, you could redirect the addresses you don't want to the null queue. Put this in your transforms.conf stanza:
REGEX=(192\.168\.0\.1|127\.0\.0\.1)
DEST_KEY=queue
FORMAT=nullQueue
Sorry for the thread necro, but I wanted to post a solution I developed today to this issue since I didn't find any examples of this anywhere else. The following negative regex lookahead excludes these 3 IP's as an example. It's very easy to modify for your own list.
IP's excluded: 127.0.0.1, 192.168.0.1, 192.168.0.254
remoteAddress = ^(?!127\.0\.0\.1|192\.168\.0\.(1|254)).+$
Implemented today via the Splunk_TA_Windows inputs.conf on a test Windows universal forwarder. Seems to be doing the trick.
One way to do a negative regex, if the filter is simple, is to use successive "not characters in set" groups. For example, to exclude traffic to LOCALHOST:
[WinNetMon://winnetmon]
....
## do not forward packets to localhost
remoteAddress="^[^1]|1[^2]|12[^7]"
So we are saying anything that doesn't start with a 1 OR starts with a 1, but not followed by a 2, OR ...
Unfortunately, regex doesn't support negation. You'll have to come up with a regex that specifies the addresses you want to see rather than those you do not want to see.
Alternatively, you could redirect the addresses you don't want to the null queue. Put this in your transforms.conf stanza:
REGEX=(192\.168\.0\.1|127\.0\.0\.1)
DEST_KEY=queue
FORMAT=nullQueue
So, just to make sure I understand completely...
In props.conf, on the indexer (not the universal forwarder), add the following:
[WinNetMon://inbound]
TRANSFORMS-null= setnull
[WinNetMon://outbound]
TRANSFORMS-null= setnull
Then in transforms.conf (again on the indexer):
[setnull]
REGEX = RemoteAddress=(192\.168\.0\.1|127\.0\.0\.1)
DEST_KEY=queue
FORMAT=nullQueue
Hi @TrevorW2000
Did @richgalloway's answer solve your question?
I am still working with this. That solution may be effective but I'm afraid the traffic would overrun the indexer in a full deployment scenario. The WinNetMon only allows a regex for RemoteAddress instead of whitelist/blacklisting. I may try to cut it off there by only logging remote hosts from certain subnets or somehow adding a range or set of ranges that would cut off that one IP address.
If I don't find an answer by the end of the month, I'll mark Rich's answer as accepted. Any further ideas though would be appreciated.
Thanks!
Yes, that should do it.