Getting Data In

Applying props on UF and transforms on the Indexer in Splunk 6

theouhuios
Motivator

Hello

I am trying to get the IIS data from windows hosts and it looks like we can apply the props.conf on the UF itself. But then I also want to apply transforms which dont work on the UF on the indexer for that sourcetype. Would it be possible to call the Transforms from the UF props itself? Or do I need props to go to both the UF and Indexers but transforms to go to just the indexers? From my understanding if the filtering and indexing is happening at UF using the props then the indexers will not try to index it again,right? If that's true how will it apply the transforms?

props

[iis]
TRANSFORMS-source_extraction = w3svc_name

transforms

[w3svc_name]
SOURCE_KEY = MetaData:Source
DEST_KEY   = MetaData:Source
REGEX      = (?i)\\(W3SVC[^\\]*)
FORMAT     = source::$1
Tags (3)
0 Karma

martin_mueller
SplunkTrust
SplunkTrust

First note it's TRANSFORMS-class, you're missing an S there.
Second, I believe those need to go on the indexer for parsing, a UF doesn't do that phase: http://wiki.splunk.com/Where_do_I_configure_my_Splunk_settings%3F

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

For rewriting the source field I'm using these without WRIE_META:

SOURCE_KEY = MetaData:Source
DEST_KEY = MetaData:Source 

Additionally, it seems as if your REGEX and FORMAT values don't really work together. Your capturing group 1 is either "es" or "gs", I guess that's not the intended source value.

0 Karma

theouhuios
Motivator

Also when I try it this way it doesn't work

[host::*WN*]
TRANSFORMS-source_extraction = source_ext

[source_ext]
SOURCE_KEY = field:source
REGEX = (es|gs)\S{1}(?P<source>[A-Z0-9]{6,7})
FORMAT = source::$1
WRITE_META = true`
0 Karma

theouhuios
Motivator

Tried this. The extraction works when i give sourcetype as microsoft_iis but not when I give sourcetype as iis. I guess thats because its a pre defined sourcetype. But the Indexed_extractions = w3c doesn't parse the fields properly when I use any sourcetype other than iis.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

The UF can and will handle a props.conf file, but it won't be able to use every single setting from it.

By "those" I'm referring to TRANSFORMS-class settings in props.conf - the transforms.conf needs to be on the indexer(s) entirely.
You can have a full copy of props.conf on both the UF and indexer(s), they will pick out the settings they can use.

theouhuios
Motivator

Missed S while pasting it here. When you say those you mean both props and transforms? Someone from Splunk actually told us that UF can handle props and it looks like it does too as it lists the props in btool.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...