Hi,
I have splunk reading from a farm of syslog servers. I don't control the syslog config, so I have to live with how they are written out to logs. I am able to read the logs, but I also want to know which server they came from. Is it possible to use a HOSTNAME variable to populate a field, or metadata? I tried the following, but it didn't work - ended up with a value of $HOSTNAME. This is on linux platforms.
[forwarder_host]
SOURCE_KEY = MetaData:Host
REGEX = host::(.*)
DEST_KEY = _meta
FORMAT = forwarder::$HOSTNAME
Not explaining this well...
I have files coming in - system.hostname.log - many different hostnames, as they are network devices which are sending syslogs to this server. I get the hostname for each logfile via the logfile name and host_regex. That part works fine. I want to take the name of the syslog server and put that in a field or metadata, so I can track back and see where the log message came from. The HOSTNAME variable at the system level is equal to this value. How do I get that assigned to this new field or metadata? Hope that makes sense...
I think you need this
Some changes to that answer:
transforms.conf
[include_host_name]
SOURCE_KEY = MetaData:Source
REGEX = <Write regex to extract HOSTNAME from system.HOSTNAME.log>
FORMAT = host_name::$2$3$4 log_sequence::$4
WRITE_META = true
props.conf
[<Your stanza name>]
TRANSFORMS-include = include_host_name
fields.conf
[host_name]
INDEXED=true
We have used host to populate a field by name OriginIP by setting fieldalias in props.conf
[my_sourcetype]
FIELDALIAS-OriginIP = host AS OriginIP
Try something like this
The hostname that I need isn't coming from any log - it's coming from the hostname of the server where splunk is running.
So, you need to assign the HOSTNAME that is present in log file name to all the events in that log.
Sorry - not sure what you are saying here. I should also mention that each host is gatherered from the name of the file - system.HOSTNAME.log, as these are network device syslogs. So, the hostname is the name of the network device, and I want some way to indicate the name of the server that received the syslog and where splunk processes it.
On a side note, Splunk should really have a "splunk_forwarder" field. So people can track where messages are originating from...
If your regex is going to return HOSTNAME then isn't it that you should use FORMAT = forwarder::$1
documentation says: FORMAT for index-time extractions:
* Use $n (for example $1, $2, etc) to specify the output of each REGEX match.
So if REGEX has only one capturing group and its value is "bar", then:
* "FORMAT = foo$1" yields "foobar"
* "FORMAT = foo$bar" yields "foo$bar"
* "FORMAT = foo$1234" yields "foo$1234"
* "FORMAT = foo$1\$2" yields "foobar\$2"