split based on lookup

[coenvandijk]

Hello,

I have a set of windows events (4656 and 4663) which contain fullpathnames. I also have a list of 'critical' directories as a lookup. I would like to split the fullpathname field into 2 new fields: One containing the 'critical' directory and one containing the remainder.

Example:
Full path: T:\k-share\A&O IR\Bestuursrelaties\W_BR_Archief\OPF\gedragscode\2007\brief aan A. Atmopawiro inzake gedragscode 2007.doc
Critical directory: T:\k-share\A&O IR\Bestuursrelaties\W_BR_Archief
Remainder: OPF\gedragscode\2007\brief aan A. Atmopawiro inzake gedragscode 2007.doc

What is the best option to achieve this?

best regards,
Coen van Dijk

[DalJeanis · Accepted solution]

Here's a run-anywhere sample that creates a rex.

| makeresults 
| eval critical="T:\k-share\A&O IR\Bestuursrelaties\W_BR_Archief!!!!C:\George\Washington\!!!!T:\k-share\Mickey.Mouse+Minnie*Mouse" 
| makemv delim="!!!!"  critical 
| table critical 
| mvexpand critical 
| rename COMMENT as "everything above this just generates test data with each critical directory in a single field called critical"

| rename COMMENT as "now we begin to format the records into a regular expression that will find directory1 or directory 2 etc"
| format "(?i)^(?<critical>" "" "3" "" "|" ")"
| rename search as searchRex
| rename COMMENT as "slashes have already been escaped, but we have to escape * and . and +"
| rex mode=sed field=searchRex "s/([\.\*\+])/!!!\1/g"
| rex mode=sed field=searchRex "s/!!!/\\\/g"
| rename COMMENT as "kill the field name and associated quotes"
| rex mode=sed field=searchRex "s/ critical=//g"
| rex mode=sed field=searchRex "s/\"  \| \"/|/g"
| rex mode=sed field=searchRex "s/> \"/>/g"
| rex mode=sed field=searchRex "s/\"  \)/)(?<remainder>.*)$/g"
| eval searchRex = "\"".searchRex."\""

The sample rex looks like this -

"(?i)^(?<critical>T:\\k-share\\A&O IR\\Bestuursrelaties\\W_BR_Archief|C:\\George\\Washington\\|T:\\k-share\\Mickey\.Mouse\+Minnie\*Mouse)(?<remainder>.*)$" 

Now, just in case that looks overcomplicated, let me get you a simple example of how it ends up -

| rex field=fulldirectory  "(?i)^(?<critical>T:\\critical\\directory1|T:\\critical\\directory2|T:\\critical\\directory3)(?<remainder>.*)$" 

That rex will put the value of any matching directory into the field named "critical" and then put anything left over into the field "remainder".

Unfortunately, the rex command does not natively take a variable for the regular expression string, so we're going to have to use the map command to have that happen.

And, map is a bit finnicky, so for testing you will need to use head to limit the results, and probably run a few times before you get it all to work right together.

 | map search="search  ...your other search terms here... | use head 5  here for testing | rex field=fulldirectory $searchRex$ | ...remainder of your search..."

[adonio]

you can create a field for the critical directory
here is an example:
https://answers.splunk.com/answers/468028/regex-source-and-destination-files-with-path-filen.html