Splunk Search

Best Methods to Improve Performance of Dashboard

katzr
Path Finder

I have a dashboard with ~38 panels with 2 joins per panel. I'm curious what is the most costly for Splunk performance of a dashboard- is it the large number of panels I have or is it the number of joins I have in each?

What are some common ways to improve the performance of a dashboard? Below is an example of one of my panels. I am doing some weird things with my location info because using the default value setting in my lookup table was throwing me a weird error.

 index=example date_month=August date_year=2017 (assignment_group="*") | dedup number | fillnull value="UNKNOWN" location | eval regionblank= "UNKNOWN" | eval countryblank= "UNKNOWN" | eval locationblank="UNKNOWN" | lookup CurrentSiteInfo.csv location| eval site=coalesce(location2,locationblank) | eval Region=coalesce(Region,regionblank)| eval Country=coalesce(Country,countryblank) | search ((Region="*") (Country="*") (site="*")) |stats count as Tickets by contact_type | join overwrite=false contact_type [search index=example earliest="6/01/2017:00:00:00" latest="12/31/2017:24:00:00" (assignment_group="*") | dedup number | fillnull value="UNKNOWN" location | eval regionblank= "UNKNOWN" | eval countryblank= "UNKNOWN" | eval locationblank="UNKNOWN" | lookup CurrentSiteInfo.csv location| eval site=coalesce(location2,locationblank) | eval Region=coalesce(Region,regionblank)| eval Country=coalesce(Country,countryblank) | search ((Region="*") (Country="*") (site="*"))| bucket _time span=1mon | stats count as Tickets by contact_type _time | stats avg(Tickets) as Baseline by contact_type | eval Baseline = round(Baseline,0)] | eval "Baseline Variance" = Tickets - Baseline | join overwrite=false contact_type [search index=example earliest=-3mon@mon (assignment_group="*") | dedup number | fillnull value="UNKNOWN" location | eval regionblank= "UNKNOWN" | eval countryblank= "UNKNOWN" | eval locationblank="UNKNOWN" | lookup CurrentSiteInfo.csv location| eval site=coalesce(location2,locationblank) | eval Region=coalesce(Region,regionblank)| eval Country=coalesce(Country,countryblank) | search ((Region="*") (Country="*") (site="*")) | bucket _time span=1mon | stats count as Tickets by contact_type _time | stats avg(Tickets) as Average by contact_type | eval Average = round(Average,0)] | eval "Average Variance" = Tickets - Average | table contact_type Tickets Baseline "Baseline Variance" Average "Average Variance" | addcoltotals | sort 0 Tickets
0 Karma

DalJeanis
SplunkTrust
SplunkTrust

Okay, I know we've already reviewed and simplified that search for you.

Ah, you commented on it an hour ago. We've given you that code over there.

https://answers.splunk.com/answers/578622/how-to-make-my-search-more-efficient-help-to-remov.html

Now, if you'd let us know how the different searches are different from each other, then we can discuss how to set up a base search + postprocessing in order to meet your need.

0 Karma

katzr
Path Finder

Hi @DalJeanis I appreciate all of your help. A lot of the searches are the same format as the one above, but are looking at it from tickets by region, problem, category, etc. instead of contact_type. But the baseline and average parameters are the same.

0 Karma

adonio
Ultra Champion

ask yourself if 38 panels are really necessary.
you are tying 38 cores when you are starting this dashboard...
there are plenty of other tricks like base search and more

0 Karma

MuS
SplunkTrust
SplunkTrust

Additionally to this advice, use a base search and post process the results http://docs.splunk.com/Documentation/Splunk/latest/Viz/Savedsearches#Post-process_searches_2
As base search use a search that skips the join and use stats instead, read here http://sideviewapps.com/slides/2017_05_02_sideview_let_stats_sort_them_out.pptx to get some create advice on how to use stats or here https://answers.splunk.com/answers/129424/how-to-compare-fields-over-multiple-sourcetypes-without-jo...

cheers, MuS

0 Karma

katzr
Path Finder

This dashboard is for a customer who requested this high number of panels. Can you expand upon the "plenty of other tricks"

0 Karma

adonio
Ultra Champion

so really depends on the use cases and on what the panels cover.
you can create base searches, improve searches, set panels loading order look here: https://answers.splunk.com/answers/513660/how-to-set-loading-order-for-panels.html
regardless, 38 panels is plenty, try and sort out the use cases and maybe split this dashboard to 5-6 dashboards.
you can always increase the cpu on the system if this is a possibility
hope it helps

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...