Splunk Search

Routing the data to a different Index via Regex

shiv1593
Communicator

Hello,

We have a source ABC sending us logs and are being stored inside an index called all_logs. From that source, we want to separate the events, which contains the field SiteUrl: https://www.abc.com/site/PathologyPHI and send these logs to a new index called new_logs, while the rest of the logs should still go to the Index all_logs.

This is what I've tried so far, but all the logs still land up in the Index all_logs, instead of being routed.. FYI, props and transforms are kept under $SPLUNK_HOME/etc/system/local, while the inputs.conf is under the TA, which brings the logs's local directory. Had to do this because in props, there is a setting defined for another sourcetype as well. Any suggestions regarding moving them back to TA as well will be appreciated.

In props.conf, I made the following change:

[abc:management:activity]
TRANSFORMS-routing = abc_logs

In Transforms.conf, I did this:

[abc_logs]
REGEX = SiteUrl:.+PathologyPHI
DEST_KEY = _MetaData:Index
FORMAT = new_logs

Here is the inputs.conf for the source. There are other inputs too, but this inputs comes with the events, which we want to route. We do not want to route all the events coming off this input, only the ones matching the regex:

[splunk_ta_abc_management_activity://Auditabc]
content_type = Audit.abc
index = all_logs
sourcetype = abc:management:activity
interval = 120
tenant_name = ABC
disabled = 0
start_by_shell = false

Any help will be highly appreciated.

Thank you

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @shiv1593,
at first, please share again the regex in transforms.conf using the Code Sample button because it isn't possible to read it and check the regex.

Anyway, where do you have props.conf and trasnforms.conf?
they must be on Indexers or (when present) on Heavy Forwarders.

Ciao.
Giuseppe

0 Karma

shiv1593
Communicator

Hi Giuseppe,

Please find the regex below:

REGEX = SiteUrl:.+PathologyPHI

Props and transforms are in the heavy forwarder under /etc/system/local. That's where I'm trying to route the data from.

Thank you

0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @shiv1593,
the regex and your approach seem to be correct.
So please some additional info:

A surely stupid question: did you restarted Heavy Forwarder after update?

Logs are in an Universal Forwarder (read by the inputs.conf in the TA) and they are sent to the HF (where there are props.conf and transforms.conf) and then sent to Indexers, is it correct or there's anything else?

I don't understand what you say: "...setting defined for another sourcetype...".

Ciao.
Giuseppe

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...