Hi
How can I extract these fieldnames and values from this event?
Step: 0345
Result: Valid
Step: 3345
Result: Valid
Field names and values:
1. Field Name: test_0345, Field Value: Valid
2. Field Name: test_3345, Field Value: Valid
I already know how to extract both the name and value, but I don't know how to add "test_" before all extracted field names.
Edit (more info):
The raw data has only one timestamp and it is located at the beginning of the file. This the reason why I didn't divide the file in multiple events.
I have events with thousands fields so renaming is not option.
At this moment, the fields look like this:
1350=Valid
2342=Valid
2345=Failed
2341=Valid
I'm looking for a solution using transforms.conf and props.conf so these fields will be available for all search in this sourcetype.
My current transforms.conf:
REGEX=(?ms)^Step:\s*([^\r\n]+)[\r\n]+(.*?)(?=\Z|[\r\n]+Step:)
CLEAN_KEYS = false
FORMAT = $1::$2
MV_ADD = 1
Assuming that you have fields called Step
and Result
, this should work:
| eval Step = "test_" . Step | eval {Step} = Result | fields - Step Result
Because calculated fields
happen last, you should be able to put this inside your props.conf
like this:
EVAL-Step = "test_" . Step | eval {Step} = Result
But I don't have a field called result. I only have fields called 1234,1235,1236, etc. I can't divide the capturing in two fields because there is more information that just the "Valid or Failed" status. As you can see in the regex everything between "Step" words is captured. If I divide the fields I might end up with steps with wrong values or with multiple values.
Right now I am looking at the documentation and I see that there is way to concatenate a fix string using FORMAT.
http://docs.splunk.com/Documentation/Splunk/6.1/admin/Transformsconf
I have tried the following FORMAT but I haven't been successful:
FORMAT = step_$1::$2
It is not possible to concatenated fields with Format at search time:
NOTE: You cannot create concatenated fields with FORMAT at search time. That
functionality is only available at index time.
Right, change to this and then it should all work (but you will have "extra" fields as a result):
FORMAT = Step::$1 Result::$2
I managed to capture the two fields with
FORMAT = Step::$1 Result::$2
but when I added:
EVAL-Step = "test_" . Step | eval {Step} = Result
I don't see any fields. If I changed the previous EVAL to REPORT like this
Report-step_num = step_num #step_num is the name of the stanza
I see both fields, so there is a problem with the EVAL.
The syntax is special and needs the pipe, either one of which may not be supported by calculated fields
so you will have to experiment. You may end up having to do the last steps at search-time.
I updated the question with more information.
One quick way would be to re-name the fields after you have them extracted..
.. | rename 0345 AS test_0345
This isnt so efficient with large number of fields though. You can also do
.. | rename 0* AS test_0* | ..
You can wildcard match on field names. However, we cant match on regex patterns that Im aware off. So you would need to iterate through the number patterns of your fields to get this for all fields. Which again, is time consuming if you have a large number of fields..
I should have included this in my question. Renaming is not an option because I have thousands of steps. The raw data only has one timestamp at the beginning of the file and that's the reason why I didn't divide the file in multiple events. Also I need these fieldnames to be a fix name for all search.