Splunk Search

REGEX transforms.conf, NOT operator

aspa
Engager

Hi Splunkers

I'm new to splunk and currently playing around with the heavy forwarder. I found here several examples how to match a specific string in an event and only forward that event to the indexer. However, I could not find the opposite.

Here is what I would like to do.

Let's say in /var/log/messages we have those two log entries:

May 7 08:00:14 <host> <user>: Splunk, get that line!
May 7 08:01:45 <host> <user>: Splunk, miss that line!

My props.conf contains:

[source::/var/log/messages]
TRANSFORMS-set=setnull,ignore

My transforms.conf contains:

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[ignore]
REGEX = !miss
DEST_KEY = queue
FORMAT = indexQueue

I have tryed several regex for the NOT miss part. So far I have not found any solution. Again, my goal is not to match the first line. I know how that is done. I want to match everything except something with a key word. At the end I should be able to exclude known events and always receive on my indexer what is wanted or sofar unknown.

Thank you for your help

0 Karma

aspa
Engager

Hello Kristian

Thank you for your reply. I figured out where my failure was in the configuration. Your post helped me a lot!

I just post shortly my solution:

The props.conf remains almost the same as above:

[source::/var/log/messages]
TRANSFORMS-set=ignore

The transforms.conf changed the most and is now shorter:

[ignore]
REGEX = miss
DEST_KEY = queue
FORMAT = nullQueue

By sending the event matching the REGEX to the nullQueue, the event is not forwarded to the indexer.

0 Karma

kristian_kolb
Ultra Champion

Have you looked at the docs?

If you have a known event that you do NOT want, you can use a matching REGEX to send that event to the nullQueue. The rest are automatically sent to the parsingQueue.

http://docs.splunk.com/Documentation/Splunk/4.3.1/Deploy/Routeandfilterdatad#Discard_specific_events...

If you want to only keep a known set of events, you do as you did above, i.e. set the queue to nullQueue for all events, and then overwrite that with parsingQueue for the events you wish to keep.

http://docs.splunk.com/Documentation/Splunk/4.3.1/Deploy/Routeandfilterdatad#Keep_specific_events_an...

UPDATE:
Saw that you send the events to indexQueue. You might be right but you probably want the parsingQueue.
Hope this helps,

Kristian

Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...