Getting Data In

Field transformation based on source

stefan_radovano
Explorer

Hi All,

We log data from devices belonging to different customers, they are written to our syslog server in files named /data/log/CUSTOMER/site/router1.log, for example. I wanted to have a search-time field called customer with the value CUSTOMER, taken from the source filename.

I did this via the web GUI, under Settings/Fields/Field Transformation, this is what was written in $SPLUNK_HOME/etc/apps/search/local/transforms.conf:

[get_customer]
FORMAT = customer::$1
REGEX = \/data\/log\/(.*)\/site.*
SOURCE_KEY = MetaData:Source

Unfortunately nothing happens, I get no field named customer when I search. From what I can tell, the regex is correct. I also tried just "source" as SOURCE_KEY but nothing changed. Is anything wrong with my transform ?

I am also not sure how this transform is applied, is it run against log messages arriving via all indexes ?

As additional info, we are running splunk on a separate server (so basically the indexer) and we use a light forwarder on our syslog server. The transform above is done on the indexer.

Thanks,
Stefan

1 Solution

emechler_splunk
Splunk Employee
Splunk Employee

In props.conf:

[your_sourcetype]
EXTRACT = \/data\/log\/(?<customer>\w+)\/site.* in source

No need to touch transforms.conf for this.

View solution in original post

stefan_radovano
Explorer

Yes, I do have that too.

0 Karma

emechler_splunk
Splunk Employee
Splunk Employee

In props.conf:

[your_sourcetype]
EXTRACT = \/data\/log\/(?<customer>\w+)\/site.* in source

No need to touch transforms.conf for this.

martin_mueller
SplunkTrust
SplunkTrust

You could also just add in source after the end of the field extraction regex when editing it through the UI.

emechler_splunk
Splunk Employee
Splunk Employee

You can do this in SPL itself:

| extract reload=t

0 Karma

stefan_radovano
Explorer

I actually tried to do this via GUI (Fields/Field Extraction) but when I chose "source" for "Apply To", it also wanted me to specify which source. I obviously didn't want to restrict it to one particular source, didn't know what to put in there. Was I doing something wrong ?

In any case, this worked. I just had to restart splunk. Is there no way around restarting ?

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

Did you specify a REPORT-foo = get_customer entry in props.conf for that sourcetype?

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...