How to prevent this from being sent to indexer from heavy forwarder

[rcavallo]

I know there have been quite a few messages on this, but I am still confused. I am trying to configure my heavy forwarder to NOT send the following from a file that is is monitoring:

[Fri Dec 02 13:10:22 EST 2011] [java.lang.String] DEBUG: <?xml version="1.0" encoding="UTF-8"?><soapenv:Envelope xmlns:soapenv

That's just the beggining of the line, but I want to key in on the word "DEBUG" and throw out all events that start with that.

I know I have to put something in the props.conf and the transforms.conf, but I do not know exactly what to put in there. Also, can you please explain what the symbols mean in the fields you are recommending? I see lots of posts with "/+s" and stuff like this, but I don't know what the heck they mean.

-Noob

[lguinn2]

First, Splunk uses regular expression (regex) syntax. The "\s+" and similar symbols are regex syntax. There are many variants for regexes; Splunk use the PCRE flavor. There are thousands of web sites and books about regular expressions, but here is one that I like: http://www.regular-expressions.info

Once you have learned a bit about regexes, the props.conf and transforms.conf configuration files will make a lot more sense. If the sourcetype of the file you are monitoring is "java-debug-stuff", then props.conf could contain this stanza

[java-debug-stuff]
TRANSFORMS-t1 = dumpDebug

This basically says "for each event of sourcetype java-debug-stuff, do the dumpDebug transformation." So in transforms.conf, you have to define the dumpDebug transformation as a stanza:

[dumpDebug]
REGEX = DEBUG
DEST_KEY = queue
FORMAT = nullQueue

The REGEX specifies a regular expression; only events that match will be affected by the transformation. Note that you don't need a fancy regex here: you are simply looking for events with the character string DEBUG.

The DEST_KEY tells Splunk to send the matching events to a queue, and FORMAT identifies the queue: the null queue (bit bucket, trash, /dev/null ...). See also the documentation (midway down the page) titled Discard specific events and keep the rest.

[rcavallo]

ok, so your example was very helpful, yet it was missing something that was preventing it from working. Many of the examples up on ANSWERS do not show putting a [source:/"path to log file] in the props.conf. I used this website to develop the regex : http://myregexp.com/ and then I used these entries and everything worked golden. Thanks very much for the help!!

Props.conf :
[source::/export/bmadmin/bmsPRD10/log/website.out]
TRANSFORMS-t1 = setnull

transforms.conf:

[setnull]
REGEX=\bDEBUG:\s.*
DEST_KEY = queue
FORMAT = nullQueue

[rcavallo]

Thank you and I completely get it. I did some reading on REGEX and I came up with this for the transforms.conf:[setnull]
REGEX = bDEBUG.$ (NOTE there is a leading slash this page won't display it)
DEST_KEY = queue
FORMAT = nullQueue

and this for the props.conf:

[website.out]
TRANSFORMS-website.out = setnull

But it's not working.... any ideas? I think my REGEX is right. Should it not be named something like setnull ( I couldnt think of anything else) or does it need to correspond to something in the file? The name of the file that I am monitoring is website.out...