How to filter out events by regex in transforms.conf?

[LuiesCui]

Hi guys,

I'm new to Splunk and I need ur help! I was trying to discard some specific events by regex and failed. Here are the events:

07/28/2015 15:45:34.037 +0800
collection="Network Interface"
object="Network Interface"
counter="Bytes Sent/sec"
instance="Intel[R] PRO_1000 MT Network Connection"

07/28/2015 15:45:34.037 +0800
collection="Network Interface"
object="Network Interface"
counter="Bytes Received/sec"
instance="Intel[R] PRO_1000 MT Network Connection"

The events have the same collection and object. I want to discard those with the line "counter="Bytes Received/sec"" and keep the events with "counter="Bytes Sent/sec"". I set props.conf as

[source::Perfmon:Network Interface]
TRANSFORMS-null=setnull

and I don't know how to write the correct regex in transforms.conf. Could you guys help me? And is there any detailed tutorial about regex for Splunk? Thank you in advance!

[jeffland · Accepted solution]

Welcome!

Regex in splunk is straight PCRE regex, so once you learn that you are good to go with regex in splunk. Learning by doing is what I'd personally recommend, and for that you should head to regex101.com whenever you want to try some expression before putting them in splunk - it's great 🙂

In your case, you could use the following simple regex:

counter\=\"Bytes\sReceived

Because you seem to know what you are doing, I presume you've already found this explanation on nullQueue and what to put where.

[LuiesCui]

Wow it works GREAT! Thank you! Just for curious, as you mention "simple regex", is there something like "advanced regex"? And in this case, if I want the regex like "^\d+-\d+-\d+\s\d+:\d+:\d+,\", how do I write the regex?

[jeffland]

No, regex is not per se either simple or advanced (though some might find any regex advanced...). There are however different implementations of regex, as "regular expression" itself is just the concept.

Please post regex as code (either with the buttons atop the text box or by leaving a blank line and indeting by four spaces), otherwise the markup will mess it up. I am guessing you wanted to say

^\d+\-\d+\-\d+\s\d+\:\d+\:\d+\,

which means start at the beginning of the string, and then have some digits with dashes, whitespace and colons between them and a comma at the end (I've escaped every character out of habit, even those that don't need it - see here for a list of characters which need escaping).
I'm not sure what you mean with "how do I write the regex", because that's how you write them - no spaces between, straight to the point.
You can start and learn something interesting with that example at this point, which is the following simplification:

^(?:\d+[-\s:,])+

This matches exactly the same pattern as above, but it states that it wants (from the beginning of the string) a digit and then one of either a dash, a whitespace or a colon - and these two, in that order, as often as possible. Try it in regex101 to get a more detailed description and to compare the two regexes!
The only drawback is that the second is much less precise, meaning that the following string:

1,15 99-106 something_further

(which would of course not be matched by the first regex) is gladly accepted by the second.

But with that, I'll let you wander off to the worlds of regular expressions to make your own discoveries. Feel free to come back any time if you have further questions 🙂