How to filter certain events so they are not returned?

[Szethius]

Our team is trying to filter out events that occur with certain tags in them. For example:

[19/Mar/2013:23:59:57 -0400] "GET /favicon.ico HTTP/1.1" 404 10607 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:19.0) Gecko/20100101 Firefox/19.0

I want to Splunk to not return any logs with "favicon.ico" in it. How would we go about doing this? Our goal is to have Splunk only return data that is relevant to finding issues and not the data we consider junk. Much thanks!

[bmacias84 · Accepted solution]

This can take place on the indexer or Heavy Forwarder using a props.conf and tranforms.conf. Will prevent any event contain the GET\s/favicon.ico from being indexed at index time.

transforms.conf

[removefavicon]

REGEX = GET\s/favicon.ico

DEST_KEY = queue

FORMAT = nullQueue

#iisw3c is is the source type of your IIS weblogs.  Though you many have change that to IIS or something else to match your sourcetype.

[iisw3c]

pulldown_type = true

MAX_TIMESTAMP_LOOKAHEAD = 32

SHOULD_LINEMERGE = False

CHECK_FOR_HEADER = False

TZ = GMT

REPORT-iisw3cfields = iisw3cfields

#Entry below is what you really want

TRANSFORMS-removefavicon = removefavicon

Alternativly you could use REPORT-<yourtransform> = <yourtransform> settings to remove it from search time.

Hope this helps or gets you started. dont forget to accept answers or vote them up if they help.

my other post:
how-to-index-w3c-iislog-from-a-universal-forwarder

[Szethius]

Before indexing since we want the filter to be permanent as we find more stuff to filter out.

[BlackZao]

Add "NOT favicon.ico" to the search string or Alt-click on the tag you don't want to have Splunk do that for you.

[Szethius]

I need something that would essentially be permanent for when I go to review logs. I need each tag that we do not need to not show up at all.

[bmacias84]

Do you want to do this before or after indexing?