Copy the value of a metadata filed to a new field at Index time

[MedralaG]

I have some network devices sending logs to a syslog server that has a UF installed. The 'host' field is populated with the name of the UF host.
We want the 'host' to be overwritten with the name of the hosts that are generating the logs initially (easy) however we also want to create an additional field ('forwarder') which will keep the name of the UF host.

I believe the best way of doing this would be to use transforms to first copy the value of the 'host' field to the new 'forwarder' field however It doesn't seem to be working with the following stanza

props.conf
[source::/var/log/messages]
TRANSFORMS-t1=copy_host_name

transforms.conf
[copy_host_name]
SOURCE_KEY = MetaData::host
REGEX = (.*)
FORMAT = forwarder::$1
WRITE_META = true

fields.conf (on the search head)
[forwarder]
INDEXED=true

any suggestions?

[MedralaG · Accepted solution]

Just for the record, I managed to find the problem that was causing my stanza in the transforms.conf not to work.
1. I had to remove the additional : in the SOURCEKEY stanza. This will allow me to pull the data from the HOST metadata field into the transforms.
2. The value pulled from the SOURCEKEY = MetaData:host will appear in the following format:

host::*hostname*

This means that I have to adjust the REGEX to exclude the host::

props.conf

[source::/var/log/messages]
TRANSFORMS-t1=copy_host_name

transforms.conf

[copy_host_name]
SOURCE_KEY = MetaData:host
REGEX = host::(.*)
FORMAT = forwarder::$1
WRITE_META = true

fields.conf (on the search head)

[forwarder]
INDEXED=true

With these corrections made I am successfully able to copy the value of the HOST field to a newly created field (at index time) and then I'm free overwrite the original host field.

[woodcock]

Good job on updating us! Now close it all off by clicking Accept on your answer.

[MedralaG]

Done, thank you.

[woodcock]

First, do your syslog this way:
http://www.georgestarcher.com/splunk-success-with-syslog/

But do it with 1 wrinkle: make sure that your syslog host name is also in the file path.
Then you can get at it any time that you like with this:

SOURCE_KEY = MetaData::source

Or something like this:

| rex field=source "(?:[^\/]+\/){4}(?<syslog_host>[^\/]+)" 

[gcusello]

Hi MedralaG,
I'd use a different approach:
I'd replace host with the correct host value (as you're doing) and I'd create a lookup with two columns:

• host of appliance sending logs to a UF,
• UF name.

In this way, you always have both the information, host and related UF, in an easier way.
Bye.
Giuseppe