I have the following data in a key (called test_key through a field extraction) I want to split:
domain\firstname.lastname|38372|VENDOR_CODE
I try to extract some values and assign them to Application and ProcessId key from another field extraction. At search time the extractions seem to work with the following:
| rex field=test_key "\|(?<ProcessId>\d+)\|" | rex field=test_key "\|(?<Application>\D+)$"
However when I define them in my transforms.conf (currently local/transforms.conf for testing) in the app I am bundling everything in I can not get them to work, this is the output as defined while entering it through the web interface:
[Application]
CLEAN_KEYS = 0
REGEX = \|(?<Application>\D+)$
SOURCE_KEY = test_key
[ProcessId]
CLEAN_KEYS = 0
REGEX = "\|(?<ProcessId>\d+)\|"
SOURCE_KEY = test_key
At search time I do not see the fields Application and ProcessId appearing, where I do see them as soon as I add the regex stanza to the search at search time. Any clues on how to get my transformations working?
How about you merge the extraction with your current EXTRACT entry , like this
EXTRACT-test_format= (?:[^\t\n]*\t){2}(?P<facility>[^\t]+)\t(?P<loglevel>[^\t]+)\t(?P<time2>[^\t]+)\t(?P<raw_serialno>[^\t]+)\t(?<test_key>([^\|]+\|(?<ProcessId>\d+)\|(?<Application>\w+)))[^\t\n]*\t(?P<unknown>[^\t]+)\t(?P<message>.+)
Can you please show your props.conf? Depending on that, it might be possible that test_key
is extracted AFTER your new extractions - therefore it doesn't work.
I doubt that, the node field is in the first extraction in my apps props.conf field.
[test_format]
category = Custom
description =
disabled = false
pulldown_type = true
EXTRACT-test_format= (?:[^\t\n]*\t){2}(?P<facility>[^\t]+)\t(?P<loglevel>[^\t]+)\t(?P<time2>[^\t]+)\t(?P<raw_serialno>[^\t]+)\t(?P<test_key>[^\t]+)[^\t\n]*\t(?P<unknown>[^\t]+)\t(?P<message>.+)
...
Do you have REPORT references in your props.conf to your transforms.conf stanzas? Transforms.conf stuff doesn't work on its own 🙂
That would definetely be something to check. 🙂
Also, EXTRACT happens before REPORT, but they're sorted by ASCII-order, not by the order they appear in the file.
Therefore (just as an example), an EXTRACT-test_format
would happen after an EXTRACT-application
, and it would also happen after an EXTRACT-ZZZ
, because uppercase letter come before lowercase in ASCII. Just want to make sure.
For more infos on search-time sequence order, check this:
http://docs.splunk.com/Documentation/Splunk/latest/Knowledge/Searchtimeoperationssequence