Knowledge Management

is it right to use summary index to save non-statistical data?

RiccardoV
Communicator

Hi,
I'm using summary index, but I am not sure if I'm doing it right.
I have several searches that save data into my summary index. Some of them are saving statistical data, ie. how many events for each category I found.
But I need to speed up some different queries, where I need to display a table with many fields, ie. md5 - process - timestamp - category - etc.

Is it correct to run a scheduled query that saves data in summary index in that way?

Thanks!

edit: here is my search

sourcetype="tcp-raw" type=WSAEXEC OR type=WSAPROCESS
| eval ...
| search ...
| stats count ...
| stats dc(det) AS ...
| eval Category = ...
| table field1, field2, field3....

I did some changes, of couse 🙂

I retrieve the data in my dashboard in this way:

index="summary" 
| dedup field1, field2
| stats count by field1, field2
| sort -count
| head 10
| fields field1, field2, ....

thanks, again 🙂

edit #2:

here is my new search, only with streamable commands:

sourcetype="tcp-raw" type=WSAEXEC OR type=WSAPROCESS
| eval ...
| search ...
| stats count ...
| stats dc(det) AS ...
| eval Category = ...
| table field1, field2, field3....

I activated report acceleration with "1 month" summary range. Do I Have to leave empty start time and finish time values in time range?

How can I retrieve this "accelerated" data now? Just doing the same exact search I accelerated?

thanks!

0 Karma
1 Solution

lguinn2
Legend

You said "I activated report acceleration with "1 month" summary range. Do I Have to leave empty start time and finish time values in time range?

How can I retrieve this "accelerated" data now? Just doing the same exact search I accelerated?"

Answers:

First, you do not need to leave the start time and finish times empty in the time range. Splunk figures out how to accelerate date in the 1 month range automatically, regardless of the start and finish times.

Second, whenever you run this search, it will be accelerated. You don't need to do anything more; there are no special steps. In fact, if you run a similar search and Splunk can leverage the underlying acceleration summary, it will!

Finally, you shouldn't need to do any maintenance on the acceleration summary; Splunk will keep it valid and up to date.

View solution in original post

lguinn2
Legend

You said "I activated report acceleration with "1 month" summary range. Do I Have to leave empty start time and finish time values in time range?

How can I retrieve this "accelerated" data now? Just doing the same exact search I accelerated?"

Answers:

First, you do not need to leave the start time and finish times empty in the time range. Splunk figures out how to accelerate date in the 1 month range automatically, regardless of the start and finish times.

Second, whenever you run this search, it will be accelerated. You don't need to do anything more; there are no special steps. In fact, if you run a similar search and Splunk can leverage the underlying acceleration summary, it will!

Finally, you shouldn't need to do any maintenance on the acceleration summary; Splunk will keep it valid and up to date.

RiccardoV
Communicator

thanks again for you answers @lguinn!

0 Karma

lguinn2
Legend

A search that contains a transaction command can't be accelerated. But I still think that report acceleration might be a better way to do this. Why are you using the transaction command? I think this might be optimized - quite a lot, actually.

RiccardoV
Communicator

Edit #2: I updated again my inizial search with only streamable commands and I accelerated my search..please have a look!

0 Karma

RiccardoV
Communicator

thanks again for your reply @lguinn!
I modified my search and now I'm 100% transaction-free 🙂

I updated the search in first post!

0 Karma

lguinn2
Legend

If you want to speed up searches, but not save statistical data, use report acceleration instead of summary indexing. That said, there are rules about which searches can be accelerated.

Can you show us the actual search that you want to run?

RiccardoV
Communicator

thanks @lguinn for your answer. I've just updated my question with the searches!

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

Splunk is officially part of Cisco

Revolutionizing how our customers build resilience across their entire digital footprint.   Splunk ...

Splunk APM & RUM | Planned Maintenance March 26 - March 28, 2024

There will be planned maintenance for Splunk APM and RUM between March 26, 2024 and March 28, 2024 as ...