Splunk Search

Does transforms.conf support a regex group based rerouting?

paimonsoror
Builder

Hi all;

I am trying to build some logic for a docker/k8s integration that we are doing through fluentd. Basically we are testing one avenue of this integration utilizing fluentd -> splunk. One of the things we want to be able to do, is to leverage a field in the json payload called 'namespace' to separate the data into its own index. Right now, here is the setup of my inputs.conf

[tcp:1520]
connection_host = dns
index = app_cpceng
sourcetype = fluentd_json
acceptFrom = xx.xx.xx.xx

Now i know utilizing transforms we can do something like this:

[routemyhosttomynewindex]
SOURCE_KEY = MetaData:Host
REGEX = myhost
DEST_KEY = MetaData:Index
FORMAT = mynewindex

where in this example it reroutes the data to an alertnate index based on the hostname, but is it possible for me to do something like this:

[routemyhosttomynewindex]
SOURCE_KEY = MetaData:Raw (not sure if that is the correct meta name)
REGEX = \"namespace\":\"(\w+)\"
DEST_KEY = MetaData:Index
FORMAT = app_cpceng_$1

A very crude example above, but hopefully it illustrates what I am looking for.

0 Karma
1 Solution

somesoni2
SplunkTrust
SplunkTrust

The default SOURCE_KEY is _raw so, just exclude that in your transforms.conf stanza, ensure regex is correct and it should work fine.
Reference, (REGEX per your need)
https://answers.splunk.com/answers/246672/how-can-i-override-an-index-name-based-on-sourcety.html

View solution in original post

0 Karma

somesoni2
SplunkTrust
SplunkTrust

The default SOURCE_KEY is _raw so, just exclude that in your transforms.conf stanza, ensure regex is correct and it should work fine.
Reference, (REGEX per your need)
https://answers.splunk.com/answers/246672/how-can-i-override-an-index-name-based-on-sourcety.html

0 Karma

paimonsoror
Builder

Good to know, thanks for the info. One of my curiosities was if I could use regex group captures within the FORMAT field, but based on : https://answers.splunk.com/answers/326378/can-i-do-named-capture-in-transformsconf-format-ip.html it looks like I can, so i think my question is answered 🙂

0 Karma

sbbadri
Motivator
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...