Knowledge Management

Summary index approach with aggregate data

sistemistiposta
Path Finder

Hello,

I would like to summarize some data with aggregated statistic results. When I summarize a search like

(search statement...)    | sitimechart span=5m count(sasl_username)  sum(nrcpt)  avg(nrcpt)  by sasl_username

I can find the aggregate data in psrsvd fields. But, as I can read in http://docs.splunk.com/Documentation/Splunk/7.1.1/Knowledge/Usesummaryindexing there isn't a field for avg, so in my search the "avg(nrcpt)" is ignored.

Also, the use of psrsvd fields is unsupported. So, with summarized data I can do a search like

index=summary |  rename psrsvd_sm_nrcpt as sumDEST | table _time sumDEST | stats sum(sumDEST)

It works, but it is unsupported.

Another approach is to use collect. I tried in many way, finally my above summarizing search has the same result with

| sort _time
| streamstats time_window=5m count(sasl_username) AS NumMSG sum(nrcpt) AS NUMDST avg(nrcpt) AS MeanDST by sasl_username
| sort -_time
| bin _time span=5m
| dedup sasl_username _time
| sort _time
| table _time sasl_username NumMSG NUMDST MeanDST
| collect

...and now I preserve also the MeanDST field. But this search seems to be more complex than the equivalent with sitimechart.

What is the best approach?

Many thanks
Marco

0 Karma

sistemistiposta
Path Finder

Uhm, I now understand this doc

where I read "If you use these commands you can use the same search string that you use for the search that you eventually run on the summary index, with the exception that you use regular reporting commands in the latter search".

So I should ignore at all the summary reserved fields: for a kind of magic I just retype the same command during the indexing phase without the "si" prefix and my result appears!
It works, and also the avg metrics which doesn't have a reserved field reports the expected results!

But... let suppose I would like a complex search like this:

index=summary 
      | sort 0 _time
      | streamstats time_window=1h sum(psrsvd_sm_nrcpt) as sumXDest by sasl_username
      | timechart span=1h count(sasl_username) as "N. Msg 1h" sum(nrcpt) as "N. Dest. 1h" avg(sumXDest) as "Avg N Dest last 1h per account"  avg(nrcpt) as "Avg 5mdest last 1h"

I see no values for avg(sumXdest). To have this to work, I have to eval or rename the psrsvd_sm_nrcpt field, and put it in a table. Again, I have to switch to the previous approach.

I think to have understood power but also limits in summary index... If I want more flexibility I have to switch to unsupported approach. I'll try to avoid this.

Thank you

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...