Splunk Search

index time transform covering multiple destination indexes

juniormint
Communicator

I would like filter certain known data events into three different indexes (possibly more in the future).

Events have an embedded field called "AppName". The possible values happen to be the names of my indexes.

Is this the right approach? Does it look correct?
in transforms.conf
[RedirectToAppIndex]
REGEX = .*AppName="(?i)(app1|app2|app3)"
DEST_KEY = _MetaData:Index
FORMAT = $1

0 Karma
1 Solution

yannK
Splunk Employee
Splunk Employee

This looks ok.
make sure that the indexes app1 app2 and app3 exists with the proper case.

View solution in original post

yannK
Splunk Employee
Splunk Employee

This looks ok.
make sure that the indexes app1 app2 and app3 exists with the proper case.

yannK
Splunk Employee
Splunk Employee

yes, each event has to be compared to the regex and routed, it adds load to the indexing.

0 Karma

juniormint
Communicator

By that you just mean...index time transforms take time and resources...add complexity, etc. Right?

0 Karma

yannK
Splunk Employee
Splunk Employee

But the best solution is still to specify the index at the input level.
If you have distinct inputs per sources of course.

0 Karma

juniormint
Communicator

Thanks for calling out that they need to exist. This transform is actually part of my strategy for dealing with indexes that logically should exist but haven't yet been created for whatever reason.

All my events from a group of machines have an initial default index defined in the index meta data field (think catchall). The log events can include a preference for destination index by specifying AppName, but if it doesn't exist the log events fall back on the default index.

0 Karma

juniormint
Communicator

My goal on the matching side was to have the "(?i)" make the regex case-insensitive.

On the index name side, my experience has suggested that the ingestion does not care about case (all my indexes show up in the manager as lower case and events coming in as App1 or aPP1 all make it into app1).

Does that not line up with your experience/understanding?

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to May Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars this month. This ...

They're back! Join the SplunkTrust and MVP at .conf24

With our highly anticipated annual conference, .conf, comes the fez-wearers you can trust! The SplunkTrust, as ...

Enterprise Security Content Update (ESCU) | New Releases

Last month, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...