Splunk Search

How to write one events to multiple indexes

kmccarthy
New Member

I have many many events and they are all useful but there is a subset that is most important. I want to keep all events in the primary index and then for efficiency (dashboards and searches) create a separate index with just the important events.
(I use summary indexes for top level charts which works great but when I drill down from the chart retrieving the events is very slow - which is why I want to create the intermediary index with just the important events).

I've been trying variations of props.conf and transforms.conf to write the important events to a second indes.

props.conf

[source::/var/log/ftauditevents]

TRANSFORMS-index = ImportantEvents

transforms.conf

[ImportantEvents]

REGEX =

DEST_KEY = _MetaData:Index

FORMAT = important_index

The problem is that the important events are not written to the primary index - and I still want the primary index to contain all events for other purposes.

Any suggestion?

Tags (1)
0 Karma

FritzWittwer_ol
Contributor

I have the same requirement here, I know we will use more licences but we need some events in more than one index, did you find a solution?

0 Karma

GKC_DavidAnso
Path Finder

If we write the data to two indexes, you will be using more license than necessary.

In your situation, I would consider:

  • Can't I just move the important data to a different index and search across them both when I want all the data (index=important OR index=hohum)?
  • Why is the drilldown search so slow? Can I make the search more efficient? (Have you ordered your search so that it is in the order that excludes the most items first? Have you used |fields to avoid extracting fields you don't need?)

I hope that helps.

0 Karma
Get Updates on the Splunk Community!

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...

Introducing the 2024 Splunk MVPs!

We are excited to announce the 2024 cohort of the Splunk MVP program. Splunk MVPs are passionate members of ...