Knowledge Management

Using multiple summary indexes

sc0tt
Builder

Our reporting needs are starting to grow so I am planning on creating new summaries and would like to use best practices to manage these summaries while trying to plan ahead as best as possible. I came across another post (here) about using multiple indexes for managing summaries. Based on the answer, I plan use the same structure and create 3 separate indexes (summary_5m, summary_1h, summary_1d).

Is this a good practice? Are there any other methods that may be better?

Tags (2)
0 Karma
1 Solution

emotz
Splunk Employee
Splunk Employee

Yes, it is a good practice. The objective of summary indexing in general is to reduce the amount of data to be searched/processed by an order of magnitude. Assuming that you would be using these summaries to provide dashboards and in the future trending either month over month or year over year, you will need some level of granularity to support those use cases.

In general data that will be searched together and is of the same type should only be put into another index for security (one group can see and and others should not) or for retention (you want to keep certain data longer than others, maybe for compliance reasons). Given that, if you need to keep the less than 1 hour summary information for 90 days and the 1 hour summary for 2 years and the 1d summary forever, they should all be put into different indexes.

Happy summarizing,
reduce, reuse, recycle

View solution in original post

0 Karma

emotz
Splunk Employee
Splunk Employee

Yes, it is a good practice. The objective of summary indexing in general is to reduce the amount of data to be searched/processed by an order of magnitude. Assuming that you would be using these summaries to provide dashboards and in the future trending either month over month or year over year, you will need some level of granularity to support those use cases.

In general data that will be searched together and is of the same type should only be put into another index for security (one group can see and and others should not) or for retention (you want to keep certain data longer than others, maybe for compliance reasons). Given that, if you need to keep the less than 1 hour summary information for 90 days and the 1 hour summary for 2 years and the 1d summary forever, they should all be put into different indexes.

Happy summarizing,
reduce, reuse, recycle

0 Karma

sc0tt
Builder

Thanks for your response and insight.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...