regex used in transforms.conf isn't extracting the fields though there isn't anything worng with the regex?

[pavanae]

I have a regullar expression extracted in transforms.conf as below :-

[split_and_extract_commands]
SOURCE_KEY = abc_def
REGEX =((?[^|&#; ]\/)?(?[^|&#; ]+)( *(?[^&|#;](-exec)|[^&|#;]*))?)
MV_ADD = TRUE

For some reason the regex above isn't displayed well so please find the screenshot for the regex below which i have used in transforms.conf

I'm sure that the regex is working fine and extracting the commands as expected as I had verified as below in regex101.com

Also I can confirm that I can see the transforms extraction in Splunk GUI, Since the SOURCE_KEY in the transforms is abc_def. I also can confirm that the abc_def is extracting fine and the results of the field were the commands you can see in the screenshot above.

But still I haven't couldn't able to figure out why isn't working.

I am using splunk version 6.4.3 and I have used the same transforms in another app and see working there.

Could anyone guide me in the right direction and help me figure out the route cause?

[somesoni2 · Accepted solution]

No as the transform is done before an autoextracted field is calculated. So what you should do here is apply transform on the _raw field as the actual field abc_def is part of it as it was auto-extracted. If you can share a sample raw event, we can help ensure the regex is properly adjusted.

[pavanae]

Thank You very much for the response @somesoni2 and abc_def is an auto extract so i was thinking to re-extract the abc_def from raw with some field naming ghi_klm and modify my SOURCE_KEY in thye transforms as something like below :-

SOURCE_KEY = ghi_klm

Where some of the sample fields from raw for abc_def are as below

abc_def="sudo su"
abc_def="su -"
abc_def="exit;"
abc_def="/etc/system temp;"

Now what could be the regex that extracts all the information in the quotes after the abc_def in _raw?

I was trying to write something like below in the props.conf

[sourcetype_Name]
ghi_klm = "command between the quotes"

It would be really helpfull if can help with thi too. Thanks in Advance

[somesoni2]

Since you're extracting your new fields from transform from abc_def field which is fairly a standard format in your logs (abc_def="some value"), whey not just use _raw to in transform and avoid intermediate FE altogether. Like this (transforms.conf)

[split_and_extract_commands]
SOURCE_KEY = _raw
REGEX =abc_def=\"((?[^|&#; ]*\/)?(?[^|&#; ]+)( *(?[^&|#;]*(-exec)|[^&|#;]*))?)\"
MV_ADD = TRUE

[somesoni2]

You're trying to extract field from an existing fields. What type of field this is or in other words, how is field abc_def was extracted? The reason I asked is that the search time field extractions, which include inline/transform field extractions, calculated fields, automatic lookups etc, have specific sequence. So, if field abc_def is auto-extracted fields, then it's not available during field transforms. See this for more details on search time operation sequence.
http://docs.splunk.com/Documentation/Splunk/6.4.3/Knowledge/Searchtimeoperationssequence#Search-time...

[pavanae]

Yes abc_def is an auto extracted field. So in splunk can't we extract the fields from an auto extracted field?