Splunk Search

How to write one events to multiple indexes

kmccarthy
New Member

I have many many events and they are all useful but there is a subset that is most important. I want to keep all events in the primary index and then for efficiency (dashboards and searches) create a separate index with just the important events.
(I use summary indexes for top level charts which works great but when I drill down from the chart retrieving the events is very slow - which is why I want to create the intermediary index with just the important events).

I've been trying variations of props.conf and transforms.conf to write the important events to a second indes.

props.conf

[source::/var/log/ftauditevents]

TRANSFORMS-index = ImportantEvents

transforms.conf

[ImportantEvents]

REGEX =

DEST_KEY = _MetaData:Index

FORMAT = important_index

The problem is that the important events are not written to the primary index - and I still want the primary index to contain all events for other purposes.

Any suggestion?

Tags (1)
0 Karma

FritzWittwer_ol
Contributor

I have the same requirement here, I know we will use more licences but we need some events in more than one index, did you find a solution?

0 Karma

GKC_DavidAnso
Path Finder

If we write the data to two indexes, you will be using more license than necessary.

In your situation, I would consider:

  • Can't I just move the important data to a different index and search across them both when I want all the data (index=important OR index=hohum)?
  • Why is the drilldown search so slow? Can I make the search more efficient? (Have you ordered your search so that it is in the order that excludes the most items first? Have you used |fields to avoid extracting fields you don't need?)

I hope that helps.

0 Karma
Get Updates on the Splunk Community!

Stay Connected: Your Guide to May Tech Talks, Office Hours, and Webinars!

Take a look below to explore our upcoming Community Office Hours, Tech Talks, and Webinars this month. This ...

They're back! Join the SplunkTrust and MVP at .conf24

With our highly anticipated annual conference, .conf, comes the fez-wearers you can trust! The SplunkTrust, as ...

Enterprise Security Content Update (ESCU) | New Releases

Last month, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...