Splunk Search

Preserve host field under a new name while doing custom host extractions

HansWurscht
Path Finder

Hi,

we are receiving log data from various network devices on a syslog server. This log data is then forwarded to our splunk deployment via a installed splunkforwarder.

Our splunk-indexservers do a field extraction at index time to set the host-field to the original device, which sent the syslog message. In this step, the former value of the host field (in this case our syslog server) gets overwritten.
How can i preserve the value of the host field under a new field (preserved_host)?

I alread tried this, but it didn't work. After a restart of the splunk-indexserver no field named preserved_host shows up.

transforms.conf:

# Copy host to a new field preserved_host
[preserveOriginalHost]
SOURCE_KEY = MetaData:Host
REGEX = (.*)
FORMAT = preserved_host::"$1"
WRITE_META = true

props.conf:

# For all incoming data, preserve the host in a new field (just in case it get's overwritten by other extractions)
[(?::){0}*]
TRANSFORMS-preserveOriginalHost = preserveOriginalHost

fields.conf:

[preserved_host]
INDEXED=true

Thanks!

woodcock
Esteemed Legend

If using HF, put this on HF, if sending directly to Indexers (or through Cluster Master), put on the Indexers; you will need to restart all transforming splunk instances:

In transforms.conf:

[preserved_host_FROM_host]
SOURCE_KEY = MetaData:Host
REGEX = host::(.*)$
FORMAT = preserved_host::$1
WRITE_META = true

In props.conf:

[syslog]
TRANSFORMS-preserved_host_FROM_host = preserved_host_FROM_host
0 Karma

kml_uvce
Builder

props.conf

[source:sourcename]
TRANSFORMS-preserveOriginalHost = preserveOriginalHost,originalhost

transforms.conf

[preserveOriginalHost]
SOURCE_KEY = MetaData:Host
REGEX = (.)
FORMAT = preserved_host::$1
WRITE_META = true
[originalhost]
DEST_KEY = MetaData:Host
REGEX = (.
) \d\d:\d\d:\d\d (\w+) (.*)
FORMAT = host::$2
WRITE_META = true

fields.conf

[preserved_host]
INDEXED=true

0 Karma

fdi01
Motivator

Set the default host value
[default]
host =
Set to your chosen default host value.

Edit inputs.conf in $SPLUNK_HOME/etc/system/local/ or in your own custom
application directory in $SPLUNK_HOME/etc/apps/ .
[monitor://]
host =

0 Karma

kml_uvce
Builder

you are sending data from devices to syslog server and then sending data from this syslog server to indexer by forwarder, i this indexer is getting the hostname of your syslog server and not device hostname, is this correct ? if yes then indexer is getting hostname from forwarder and your above code will not work

0 Karma

HansWurscht
Path Finder

Hi,

we are sending syslog data from network devices to our syslog server. This data is then forwarded (via splunkforwarder) to our indexservers.

By default, the host field of this data in splunk is set to the syslog_server (host=syslog_server)
But since the log data was generated originally on the network devices, we rewrite the the host field. For example in a log data

2014 Dec 30 11:46:35 firewall-zone-a ICMP blabla

we set the host to host=firewall-zone-a.

But now i've lost the information, from which syslog_server this message came from.
So before rewriting the host field to firewall-zone-a i want to copy the value of host to a new field (preserved_host):

First(1), save current value to a new field:

preserved_host=syslog_server

Then(2), rewrite host to the log originating device:

host=firewall-zone-a

Any suggestions how to do the first step?

Thanks!

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...