Getting Data In

Universal Forwarder props.conf and transforms.conf settings

kniloo
Explorer

I am trying to get the output from a python script to indexer. So i added transforms.conf and props.conf under C:\Program Files\SplunkUniversalForwarder\etc\system\local

transforms.conf
[myexternaltable]
REGEX = (.)
external_cmd = addnum.py $1
DEST_KEY = queue
FORMAT = indexQueue

props.conf
[sitescope_daily2_log]
TRANSFORMS-runscript=myexternaltable

But its not working, can anyone please help me with correct settings needs to be done on UF.

Thanks,
Niloo

0 Karma

didatams
New Member

Just an idea.. but if you want to input data from a script.

You can put the script in the bin directory of an app, refer it in the inputs.conf.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

The props.conf and transforms.conf files should be installed on the indexer(s), not the UF.

---
If this reply helps you, Karma would be appreciated.
0 Karma

kniloo
Explorer

I have moved props.conf and transforms.conf to indexer ,but still its not working.
transforms.conf
[myexternaltable]
REGEX = (.)
external_cmd = testscript.py $1
fields_list = log
DEST_KEY = queue
FORMAT = indexQueue
WRITE_META = true

props.conf
[sitescope_daily2_log]
TRANSFORMS-runscript=myexternaltable

0 Karma

mikev
Path Finder

I know this is an older post but I believe that you should be using DEST_KEY per the documentation:

DEST_KEY = <KEY>
* NOTE: This setting is only valid for index-time field extractions.
* Specifies where Splunk software stores the expanded FORMAT results in
  accordance with the REGEX match.
* Required for index-time field extractions where WRITE_META = false or is
  not set.
* For index-time extractions, DEST_KEY can be set to a number of values
  mentioned in the KEYS section at the bottom of this file.
  * If DEST_KEY = _meta (not recommended) you should also add $0 to the
    start of your FORMAT setting.  $0 represents the DEST_KEY value before
    Splunk software performs the REGEX (in other words, _meta).
    * The $0 value is in no way derived *from* the REGEX match. (It
      does not represent a captured group.)
* KEY names are case-sensitive, and should be used exactly as they appear in
  the KEYs list at the bottom of this file. (For example, you would say
  DEST_KEY = MetaData:Host, *not* DEST_KEY = metadata:host .)

0 Karma

kniloo
Explorer

Thanks for the response.
But if we required to parse some data at UF (before sending to indexer) can't we use transforms.conf and props.conf on UF ?

if yes ,can you share the steps as well.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

What is the source of this requirement? Just because it is required does not make it possible (or correct).
The filtering you are trying to do is performed by indexers or heavy forwarders, not universal forwarders. Consider replacing the UF with a HF.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...