Splunk Search

Field Extraction Mystery

Ant1D
Motivator

Hey,

I would like to use field extraction at search time to do the following:

My source field in Splunk contains file paths. Each file path has a word that I want to extract from it and place into another field.

E.g. Source field contains a file path \helloworld\welcome\TheWord-uvwxyz.1234.log The word I want to extract is uvwxyz.

How can I achieve this? Is there a way of doing this using props.conf and/or transforms.conf?

N.B. I do not want to extract data from _raw but from the field named source.

Thanks in advance for your help

0 Karma

southeringtonp
Motivator

This will be easier to deal with if you define a permanent extraction.

In transforms.conf:

[extract-filename]
SOURCE_KEY = source
REGEX = TheWord-([^\.]+)
FORMAT = filename::$1

In props.conf:

[yoursourcetype]
REPORT-filename = extract-filename

Tweak the regex to your liking. Change the [yoursourcetype] heading to [host::yourhost] or [source::yoursource] as needed.

The fact that you are extracting from source is something of a special case, since you can be sure of having that field already populated in the index.

If your first field is not host, source, or sourcetype, then you also need to make sure that your field extractions are called in the correct order -- naming becomes important. For example REPORT-000-fullpath and REPORT-999-filename.

Ant1D
Motivator

Thanks for the info southeringtonp. I will give this a test and let you know the results.

0 Karma

hbazan
Path Finder

In My case i use this:

... | rex field=source "basefolder\\\\(?<path>(\w+\\\\)+)(?<filename>.*).log" |  

Obtaining both the filepath and the filename. For your example I'd do:

... | rex field=source "helloworld\\\\(?<path>(\w+\\\\)+)TheWord-(?<filename>.*).log" |  
0 Karma

Brian_Osburn
Builder

During your search, you can do something like this:

.. | eval extracted=ltrim(source,"\helloworld\welcome\TheWord-") | eval extracted=rtrim(extracted,".1234.log")
0 Karma

Brian_Osburn
Builder

it would extract the field "extracted" with what ever it matched.

0 Karma

Ant1D
Motivator

Would this allow me to populate another field with the extracted words? If so, would I have to keep running this search each time I want to populate another field with this data?

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...