Getting Data In

How to filter certain events so they are not returned?

Szethius
Explorer

Our team is trying to filter out events that occur with certain tags in them. For example:

[19/Mar/2013:23:59:57 -0400] "GET /favicon.ico HTTP/1.1" 404 10607 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:19.0) Gecko/20100101 Firefox/19.0

I want to Splunk to not return any logs with "favicon.ico" in it. How would we go about doing this? Our goal is to have Splunk only return data that is relevant to finding issues and not the data we consider junk. Much thanks!

Tags (3)
1 Solution

bmacias84
Champion

This can take place on the indexer or Heavy Forwarder using a props.conf and tranforms.conf. Will prevent any event contain the GET\s/favicon.ico from being indexed at index time.

transforms.conf


[removefavicon]
REGEX = GET\s/favicon.ico
DEST_KEY = queue
FORMAT = nullQueue


#iisw3c is is the source type of your IIS weblogs. Though you many have change that to IIS or something else to match your sourcetype.
[iisw3c]
pulldown_type = true
MAX_TIMESTAMP_LOOKAHEAD = 32
SHOULD_LINEMERGE = False
CHECK_FOR_HEADER = False
TZ = GMT
REPORT-iisw3cfields = iisw3cfields
#Entry below is what you really want
TRANSFORMS-removefavicon = removefavicon

Alternativly you could use REPORT-<yourtransform> = <yourtransform> settings to remove it from search time.

Hope this helps or gets you started. dont forget to accept answers or vote them up if they help.

my other post:
how-to-index-w3c-iislog-from-a-universal-forwarder

View solution in original post

bmacias84
Champion

This can take place on the indexer or Heavy Forwarder using a props.conf and tranforms.conf. Will prevent any event contain the GET\s/favicon.ico from being indexed at index time.

transforms.conf


[removefavicon]
REGEX = GET\s/favicon.ico
DEST_KEY = queue
FORMAT = nullQueue


#iisw3c is is the source type of your IIS weblogs. Though you many have change that to IIS or something else to match your sourcetype.
[iisw3c]
pulldown_type = true
MAX_TIMESTAMP_LOOKAHEAD = 32
SHOULD_LINEMERGE = False
CHECK_FOR_HEADER = False
TZ = GMT
REPORT-iisw3cfields = iisw3cfields
#Entry below is what you really want
TRANSFORMS-removefavicon = removefavicon

Alternativly you could use REPORT-<yourtransform> = <yourtransform> settings to remove it from search time.

Hope this helps or gets you started. dont forget to accept answers or vote them up if they help.

my other post:
how-to-index-w3c-iislog-from-a-universal-forwarder

Szethius
Explorer

Before indexing since we want the filter to be permanent as we find more stuff to filter out.

0 Karma

BlackZao
Explorer

Add "NOT favicon.ico" to the search string or Alt-click on the tag you don't want to have Splunk do that for you.

0 Karma

Szethius
Explorer

I need something that would essentially be permanent for when I go to review logs. I need each tag that we do not need to not show up at all.

0 Karma

bmacias84
Champion

Do you want to do this before or after indexing?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...