Splunk Search

How to ignore some events at index time?

ktn01
Path Finder

Hello,
I have to index only events that contains the string "$$log$$".
I try with a transforms like

[ignore]
REGEX = < a regex to match event not containing "$$log$$" >
DEST_KEY = queue
FORMAT = nullQueue

How can I wrote a "not match" regex (like !~ in perl)?

Thank's
Christian

0 Karma
1 Solution

somesoni2
Revered Legend

Instead of using the 'not match' regex and drop those events, you do 'match' regex and index those matching events, and drop everything else. Something on the lines of

In props.conf (keep the exact same order of transforms):

[source::/var/log/messages]
TRANSFORMS-set= setnull,setparsing

In transforms.conf:

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = \$\$log\$\$
DEST_KEY = queue
FORMAT = indexQueue

See this for more details.

http://docs.splunk.com/Documentation/Splunk/6.5.2/Forwarding/Routeandfilterdatad#Keep_specific_event...

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

Regex doesn't do NOT very well. Your best bet is write a regex that matches want you want to keep and send everything else to nullQueue. See the answer at https://answers.splunk.com/answers/111362/filtering-of-events-using-nullqueue.html.

---
If this reply helps you, Karma would be appreciated.

somesoni2
Revered Legend

Instead of using the 'not match' regex and drop those events, you do 'match' regex and index those matching events, and drop everything else. Something on the lines of

In props.conf (keep the exact same order of transforms):

[source::/var/log/messages]
TRANSFORMS-set= setnull,setparsing

In transforms.conf:

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = \$\$log\$\$
DEST_KEY = queue
FORMAT = indexQueue

See this for more details.

http://docs.splunk.com/Documentation/Splunk/6.5.2/Forwarding/Routeandfilterdatad#Keep_specific_event...

Get Updates on the Splunk Community!

More Ways To Control Your Costs With Archived Metrics | Register for Tech Talk

Tuesday, May 14, 2024  |  11AM PT / 2PM ET Register to Attend Join us for this Tech Talk and learn how to ...

.conf24 | Personalize your .conf experience with Learning Paths!

Personalize your .conf24 Experience Learning paths allow you to level up your skill sets and dive deeper ...

Threat Hunting Unlocked: How to Uplevel Your Threat Hunting With the PEAK Framework ...

WATCH NOWAs AI starts tackling low level alerts, it's more critical than ever to uplevel your threat hunting ...