Getting Data In

How to create a custom field to match a particular string?

jwelters
Explorer

I have an interesting use case, where I have a list of strings that I search for within our proxy logs to identify problem systems. This works quite well. Currently the list exists in both a CSV format, and a search Macro. The macro just specifies (phrase1 OR phrase2 OR phrase3) it's not complicated.

Recently I decided I wanted to add to our analysis and return the results but also figure out how to add a new field that is the particular string that was matched. For example if the phrase was infected, I could display a table with the _time, URL , and matched_string. This would make analysis a bit easier for us.

I've seen a couple ways folks have looked at doing something similar but I wasn't able to find a working solution for me, has anyone done something like this before ?

0 Karma
1 Solution

somesoni2
SplunkTrust
SplunkTrust

How many keywords are there? I'm assuming there are low as you are using them in Macro. If that's still the case, you could write another macro, which will do like this. You'd end up maintaining two macros and have to update both when you add a new phrase.

 eval matched_string=case(searchmatch("phrase1"),"phrase1",searchmatch("phrase2"),"phrase2", searchmatch("phrase3"),"phrase3",...,1=1,"PhraseNotInMacro")

View solution in original post

aaraneta_splunk
Splunk Employee
Splunk Employee

@jwelters - Did one of the answers below help provide a solution your question? If yes, please click “Accept” below the best answer to resolve this post and upvote anything that was helpful. If no, please leave a comment with more feedback. Thanks.

0 Karma

DalJeanis
SplunkTrust
SplunkTrust

Here's a sample of what you can do. This makes a list (you would substitute your lookup table here)

|makeresults |eval mylist="infected disturbed reallycrazy zombiefied"|makemv mylist | mvexpand mylist 

this turns it into a tight regex

|rename mylist as search
|format "(?<resultfield>" "" 3 "" "|" ")" 
| rex field=search mode=sed "s/\>  /\>/g" 
| rex field=search mode=sed "s/  \|  /|/g"

The return value looks like this -

(?<resultfield\>infected|disturbed|reallycrazy|zombiefied )

and you can drop that into a rex command

| rex field=_raw max_match=5 [the above search using your lookup table as input]

to extract which of the lookup item(s) matched.

0 Karma

somesoni2
SplunkTrust
SplunkTrust

How many keywords are there? I'm assuming there are low as you are using them in Macro. If that's still the case, you could write another macro, which will do like this. You'd end up maintaining two macros and have to update both when you add a new phrase.

 eval matched_string=case(searchmatch("phrase1"),"phrase1",searchmatch("phrase2"),"phrase2", searchmatch("phrase3"),"phrase3",...,1=1,"PhraseNotInMacro")

jwelters
Explorer

There are a few hundred.

0 Karma

jwelters
Explorer

This was the best solution, it just wasn't a great one but it worked. I believe it's the best solution given the limitations of the core tool.

0 Karma

somesoni2
SplunkTrust
SplunkTrust

My bet will be on creating a macro with case expressions similar to above, assuming the phrase can occur at any place in the raw data (no fixed position).

jwelters
Explorer

It looks like this will work well, as there's already a script that creates the Search macro, and the lookup table... it's fairly easy to also have it create the macro for this.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...