Best Methods to Improve Performance of Dashboard

[katzr]

I have a dashboard with ~38 panels with 2 joins per panel. I'm curious what is the most costly for Splunk performance of a dashboard- is it the large number of panels I have or is it the number of joins I have in each?

What are some common ways to improve the performance of a dashboard? Below is an example of one of my panels. I am doing some weird things with my location info because using the default value setting in my lookup table was throwing me a weird error.

 index=example date_month=August date_year=2017 (assignment_group="*") | dedup number | fillnull value="UNKNOWN" location | eval regionblank= "UNKNOWN" | eval countryblank= "UNKNOWN" | eval locationblank="UNKNOWN" | lookup CurrentSiteInfo.csv location| eval site=coalesce(location2,locationblank) | eval Region=coalesce(Region,regionblank)| eval Country=coalesce(Country,countryblank) | search ((Region="*") (Country="*") (site="*")) |stats count as Tickets by contact_type | join overwrite=false contact_type [search index=example earliest="6/01/2017:00:00:00" latest="12/31/2017:24:00:00" (assignment_group="*") | dedup number | fillnull value="UNKNOWN" location | eval regionblank= "UNKNOWN" | eval countryblank= "UNKNOWN" | eval locationblank="UNKNOWN" | lookup CurrentSiteInfo.csv location| eval site=coalesce(location2,locationblank) | eval Region=coalesce(Region,regionblank)| eval Country=coalesce(Country,countryblank) | search ((Region="*") (Country="*") (site="*"))| bucket _time span=1mon | stats count as Tickets by contact_type _time | stats avg(Tickets) as Baseline by contact_type | eval Baseline = round(Baseline,0)] | eval "Baseline Variance" = Tickets - Baseline | join overwrite=false contact_type [search index=example earliest=-3mon@mon (assignment_group="*") | dedup number | fillnull value="UNKNOWN" location | eval regionblank= "UNKNOWN" | eval countryblank= "UNKNOWN" | eval locationblank="UNKNOWN" | lookup CurrentSiteInfo.csv location| eval site=coalesce(location2,locationblank) | eval Region=coalesce(Region,regionblank)| eval Country=coalesce(Country,countryblank) | search ((Region="*") (Country="*") (site="*")) | bucket _time span=1mon | stats count as Tickets by contact_type _time | stats avg(Tickets) as Average by contact_type | eval Average = round(Average,0)] | eval "Average Variance" = Tickets - Average | table contact_type Tickets Baseline "Baseline Variance" Average "Average Variance" | addcoltotals | sort 0 Tickets

[DalJeanis]

Okay, I know we've already reviewed and simplified that search for you.

Ah, you commented on it an hour ago. We've given you that code over there.

https://answers.splunk.com/answers/578622/how-to-make-my-search-more-efficient-help-to-remov.html

Now, if you'd let us know how the different searches are different from each other, then we can discuss how to set up a base search + postprocessing in order to meet your need.

[katzr]

Hi @DalJeanis I appreciate all of your help. A lot of the searches are the same format as the one above, but are looking at it from tickets by region, problem, category, etc. instead of contact_type. But the baseline and average parameters are the same.

[adonio]

ask yourself if 38 panels are really necessary.
you are tying 38 cores when you are starting this dashboard...
there are plenty of other tricks like base search and more

[MuS]

Additionally to this advice, use a base search and post process the results http://docs.splunk.com/Documentation/Splunk/latest/Viz/Savedsearches#Post-process_searches_2
As base search use a search that skips the join and use stats instead, read here http://sideviewapps.com/slides/2017_05_02_sideview_let_stats_sort_them_out.pptx to get some create advice on how to use stats or here https://answers.splunk.com/answers/129424/how-to-compare-fields-over-multiple-sourcetypes-without-jo...

cheers, MuS

[katzr]

This dashboard is for a customer who requested this high number of panels. Can you expand upon the "plenty of other tricks"

[adonio]

so really depends on the use cases and on what the panels cover.
you can create base searches, improve searches, set panels loading order look here: https://answers.splunk.com/answers/513660/how-to-set-loading-order-for-panels.html
regardless, 38 panels is plenty, try and sort out the use cases and maybe split this dashboard to 5-6 dashboards.
you can always increase the cpu on the system if this is a possibility
hope it helps