Splunk Search

Filter based on groups of keys

machosplunker
Explorer

I have logs which contains keys like this.

Concept1

key=/UUID:uuid1/concept1:100
key=/UUID:uuid2/concept1:123
..
key=/UUID:uuid3/concept1:456

Concept 2

key=/UUID:uuid1/concept2:200
key=/UUID:uuid2/concept2:201
..
key=/UUID:uuid3/concept2:202

Concept 3

key=/UUID:uuid3/concept3:301
key=/UUID:uuid4/concept3:301
..
key=/UUID:uuid5/concept3:302

Is it possible to filter the results to filter only keys containing concept1.

I am thinking about using something like key="\/UUID:/concept1:" would this work?

Tags (3)
0 Karma
1 Solution

jonuwz
Influencer

This would work.

... | where match(key,"/UUID:\w+/concept1") | ...

but this would pull back all events matching the 1st part of the search ( the ... ) then filter everything else out.

It would be more efficient to do :

concept1 | where match(key,"/UUID:\w+/concept1") | ...

so this pulls out everything that contains concept1 (its a word - splunk works on words in the initial search - where a word is something surrounded by whitespace or punctuation), And then it would filter more specifically.

or maybe just this would suffice.

key=*concept1

View solution in original post

jonuwz
Influencer

This would work.

... | where match(key,"/UUID:\w+/concept1") | ...

but this would pull back all events matching the 1st part of the search ( the ... ) then filter everything else out.

It would be more efficient to do :

concept1 | where match(key,"/UUID:\w+/concept1") | ...

so this pulls out everything that contains concept1 (its a word - splunk works on words in the initial search - where a word is something surrounded by whitespace or punctuation), And then it would filter more specifically.

or maybe just this would suffice.

key=*concept1

bjoernjensen
Contributor

I think the best way to filter data before index time is discribed here

Filtering could also be done within a heavy forwarder. Just to reduce your workload asap. So mapping to your context:

Keep specific events and discard the rest

(1) In props.conf:

[source::]
TRANSFORMS-set= setnull,setparsing

(2) In transforms.conf:

[setnull]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

[setparsing]
REGEX = key=/UUID:some_string_value/concept1:some_number_value
DEST_KEY = queue
FORMAT = indexQueue

0 Karma

machosplunker
Explorer

I edited the question, might not have been clear with what i needed.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...