Splunk Search

Why are field transformations not capturing from source field?

thenino
Loves-to-Learn Lots

I am trying to create a new field called collection which is extracted from the existing source field. I am able to extract the field during an adhoc search, but want to create it using the field transformations without having to generate a regex during each search.

The source field value is just a path (ex: source=D:\Logs\SomeCollectionName_SomeDomain\SomeDomain.log) and I am extracting part of that path to aggregate by collection. I want to specifically want to target the source field to regex not _raw.

  • BEFORE Extraction: source=D:\Logs\SomeCollectionName_SomeDomain\SomeDomain.log
  • AFTER Extraction: collection=SomeCollectionName

alt text

I have created a field transformation called testcollection and nothing is being extracted at search time.

Here are my settings for the testcollection field transformations (permissions are set for everyone to be able to read in the search app)
alt text

Maybe I have a misunderstanding of Field Transformations and should be using field extractions any guidance would be helpful. I am just using the default formatting, but maybe that is incorrect.

Note:

I have gone through the documentation for field transformations and field extracts. I understand how to extract new fields during a search, but I want this new field to be available to all of the users in our account.

0 Karma

marycordova
SplunkTrust
SplunkTrust

Use and extraction for this instead of a transformation:

alt text
alt text

@marycordova
0 Karma

thenino
Loves-to-Learn Lots

I used both field extractions and field transformations. The problem is _raw does not include the source field. I am able to parse anything that matches the regex in _raw but it is not parsing the actual source field. I want to be able to target the source field like I can during adhoc searches example:
source=D:\Logs\MyCollection_SomeDomain\SomeDomain.log

Some search
|rex field=source "(?<CollectionMap>\w+)\_"

Expected: CollectionMap=MyCollection
Actual: Anything with an underscore in _raw matches

0 Karma

marycordova
SplunkTrust
SplunkTrust

Do you have access directly to the props.conf file instead of using the UI?

@marycordova
0 Karma

thenino
Loves-to-Learn Lots

We are using splunk cloud so no access to the props.conf

0 Karma

thenino
Loves-to-Learn Lots

I think I found my answer here:
https://answers.splunk.com/answers/149597/im-struggling-with-how-i-should-be-doing-inputs-and-also-p...

I will have to open a support ticket with splunk. Thanks for your assistance.

0 Karma

thenino
Loves-to-Learn Lots

Thank you. I tried that at first and I was running into this issue (I had to disable my json browser extension)

Your entry was not saved. The following error was reported: SyntaxError: Unexpected token < in JSON at position 0.

https://answers.splunk.com/answers/106487/your-entry-was-not-saved-the-following-error-was-r.html

Once I disabled my extension I was able to create the field extraction. I will report back if it works as expected.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...