I failed to get parameter value from Splunk regex query result

[splunkusera]

Per this document in splunk (http://docs.splunk.com/Documentation/Splunk/latest/Data/overridedefaulthostassignments), i'd like to change value of a default field, "host" during index time.

There are two fields, fielda="vala" and fieldb="valb" from log event data I need to refer to.
And format of new host value is FORMAT = "host::$1.$2".

I worte 3 regex:

REGEX = fielda="(?P<AAA>[^"]*)"|fieldb="(?P<BBB>[^"]*)"

$2 is always "" and host is set to 'vala.'.

REGEX = fielda="(?P<AAA>[^"]*)".*(fieldb="(?P<BBB>[^"]*)")?

$2 is always "" and host is set to 'vala.'.

REGEX = fielda="(?P<AAA>[^"]*)".*fieldb="(?P<BBB>[^"]*)"

I got expected value, "vala.valb".

In summary, splunk regex performs non-greedy matching. How can I match all paths and get $2 filled?

Thanks,
Kevin

[splunkusera]

We finally resolve the issue with another regular expression. The expression is
host="(?[^"])".*domain="(?[^"])"|host=(?[^"]*)"

The reason we can match is for NFA engine Splunk uses, we will match alternatives one by one if there are multiple alternatives. That is it tries to match first alternative first and then second with order.

Thanks a lot for all your help, Splunk guys.

[sowings]

Since you're building up a host:: tag based upon the matches from the REGEX, I nominally thought of that as "fielda=host" and "fieldb=domain". Based upon that, I wrote up this tiny sample:

Mar 20 22:03:49.335 [RATE] host="bogfly"
Mar 21 10:03:19.335 [RATE] host="wibblenog" domain="bar.com"

I then used this regex to capture appropriately:

host="(?<AAA>[^"]*)".*(domain="(?<BBB>[^"]*)")?"

The BBB value is now in $3. The whole match for fieldb (in this case, "domain") is now optional.

[splunkusera]

I got two host values.
One is my pc hostname which is for the first event.
The other is "wibblenog." which is for the second one.
BTW, I test it in splunk 4.3.1 evaluation version in windows 7.

[splunkusera]

Still failed:-). Is there anything wrong with my configuration?

In props.conf I add following section.
[HostRewrite]
SHOULD_LINEMERGE = false
TRANSFORMS-ZYHostTrans = ZYHostTrans
NO_BINARY_CHECK = 1
pulldown_type = 1

In transforms.conf I add following section:
[ZYHostTrans]
REGEX = host="(?[^"])".(domain="(?[^"]*)")?"
FORMAT = host::$1.$3
DEST_KEY = MetaData:Host

My log content is as following:
Mar 20 22:03:49.335 [RATE] host="bogfly"
Mar 21 10:03:19.335 [RATE] host="wibblenog" domain="bar.com"

[Ayn]

It's not performing non-greedy matching, but it's true that once a complete match is identified it stops. So if you have an OR type regex with (match1|match2) it will indeed not match match2 if it's already found match1. This is not something Splunk specific really.

What's the problem with your third regex that makes you unable to use that? It looks pretty much like what I had suggested if you hadn't already created the correct regex yourself. 🙂
If the issue is that fieldb can occur before fielda, just write a regex to cover that case as well.

[splunkusera]

Thank you for your kindly help first. The problem for me is there might be only a fielda in log event without fieldb. So, I can not use regex 3 and we have order assumption for regex 2. The best way for me to use is regex 1.

Can we specific some option to perform a full match?I use RegexBuddy to test regex1 and it can highlight all result.