I have a field with search terms in the log that I want to break into individual words. But I do not want stop words (like A, AN, THE) etc to show up in the result.
eg: my log file has
searchTerm=lost+cat
searchTerm=lost+a+cat
searchTerm=lost+the_dog
And I want to get a list of all the words ever used in the searchTerm field.
lost
cat
dog
But I do not want "a" , "an", "the" to appear in the list.
I use makemv delim=" "
searchTerm | eval n = urldecode(searchTerm) | makemv delim=" " setsv=true n| top limit=10000 n
But I get all the words including "a", "the" etc. How do I get to NOT display these stop words.
You could add in a where command before | top limit=10000 n
Like this
| where lower(n)!="a" AND lower(n)!="an" AND lower(n)!="the" | top limit=10000 n
If there are many words to eliminate, then you might want to use lookups instead.
You could add in a where command before | top limit=10000 n
Like this
| where lower(n)!="a" AND lower(n)!="an" AND lower(n)!="the" | top limit=10000 n
If there are many words to eliminate, then you might want to use lookups instead.
Thank You , It worked !