Splunk Search

How to remove some words from multivalued field for display

manjushan
Explorer

I have a field with search terms in the log that I want to break into individual words. But I do not want stop words (like A, AN, THE) etc to show up in the result.

eg: my log file has

searchTerm=lost+cat
searchTerm=lost+a+cat

searchTerm=lost+the_dog

And I want to get a list of all the words ever used in the searchTerm field.
lost
cat
dog

But I do not want "a" , "an", "the" to appear in the list.

I use makemv delim=" "

searchTerm | eval n = urldecode(searchTerm) | makemv delim=" " setsv=true n| top limit=10000 n

But I get all the words including "a", "the" etc. How do I get to NOT display these stop words.

0 Karma
1 Solution

lguinn2
Legend

You could add in a where command before | top limit=10000 n

Like this

| where lower(n)!="a" AND lower(n)!="an" AND lower(n)!="the" | top limit=10000 n

If there are many words to eliminate, then you might want to use lookups instead.

View solution in original post

lguinn2
Legend

You could add in a where command before | top limit=10000 n

Like this

| where lower(n)!="a" AND lower(n)!="an" AND lower(n)!="the" | top limit=10000 n

If there are many words to eliminate, then you might want to use lookups instead.

manjushan
Explorer

Thank You , It worked !

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...