XML menu tuning

[jamesdon]

I am trying to get my menus to load faster. It seems like the searches are searching over all time, because it was really fast when I first started collecting data. I tried to move the Time Picker to the top of the search, but that didn't seem to help it load any faster.

Does anyone have advice on how to speed them up?

<?xml version="1.0"?>
<view onunloadCancelJobs="False" autoCancelInterval="100">
  <!--  autoCancelInterval is set here to 100  -->
  <label>Active VPN users</label>
  <module name="AccountBar" layoutPanel="appHeader"/>
  <module name="AppBar" layoutPanel="navigationHeader"/>
  <module name="Message" layoutPanel="messaging">
    <param name="filter">*</param>
    <param name="clearOnJobDispatch">False</param>
    <param name="maxSize">1</param>
  </module>

 <module name="SearchBar" layoutPanel="splSearchControls-inline">
    <param name="label">Search</param>
    <param name="default">*</param>
    <param name="useOwnSubmitButton">False</param>

<!-- HiddenIntention that inserts index="vpn_access". -->
<module name="HiddenIntention">
  <param name="intention">
    <param name="name">addterm</param>
    <param name="arg">
      <param name="index">vpn_access</param>
    </param>
    <!-- tells the addterm intention to put our term in the first search clause no matter what. -->
    <param name="flags"><list>indexed</list></param>
  </param>

  <!-- Search to build the drop down menu items. -->
  <module name="SearchSelectLister" layoutPanel="splSearchControls-inline">
    <param name="label">VPN Concentrator</param>
    <param name="settingToCreate">element_name</param>
    <param name="search">index=vpn_access element_name=* | stats count by element_name | sort element_name</param>
    <param name="searchWhenChanged">True</param>
    <param name="staticFieldsToDisplay">
      <list>
        <param name="label">ALL</param>
        <param name="value">*</param>
      </list>
    </param>
    <param name="selected">ALL</param>
    <param name="searchFieldsToDisplay">
      <list>
        <param name="label">element_name</param>
        <param name="value">element_name</param>
      </list>
    </param>
    <module name="ConvertToIntention">
      <param name="settingToConvert">element_name</param>
      <param name="intention">
        <param name="name">addterm</param>
        <param name="arg">
          <param name="element_name">$target$</param>
        </param>
        <!-- tells the addterm intention to put our term in the first search clause no matter what. -->
        <param name="flags"><list>indexed</list></param>
      </param>

      <!-- Search to build the drop down menu items. -->
      <module name="SearchSelectLister" layoutPanel="splSearchControls-inline">
        <param name="label">Group Policy</param>
        <param name="settingToCreate">group_policy</param>
        <param name="search"> | stats count by group_policy | sort group_policy</param>
        <param name="applyOuterIntentionsToInternalSearch">True</param>
        <param name="staticFieldsToDisplay">
          <list>
            <param name="label">ALL</param>
            <param name="value">*</param>
          </list>
        </param>
        <param name="searchFieldsToDisplay">
          <list>
            <param name="label">group_policy</param>
            <param name="value">group_policy</param>
          </list>
        </param>
        <module name="ConvertToIntention">
          <param name="settingToConvert">group_policy</param>
          <param name="intention">
            <param name="name">addterm</param>
            <param name="arg">
              <param name="group_policy">$target$</param>
            </param>
              <!-- tells the addterm intention to put our term in the first search clause no matter what. -->
              <param name="flags"><list>indexed</list></param>
          </param>

          <!-- Search to build the drop down menu items. -->
          <module name="SearchSelectLister" layoutPanel="splSearchControls-inline">
            <param name="label">Username</param>
            <param name="settingToCreate">username</param>
            <param name="search"> | stats count by username | sort username</param>
            <param name="applyOuterIntentionsToInternalSearch">True</param>
            <param name="staticFieldsToDisplay">
              <list>
                <param name="label">ALL</param>
                <param name="value">*</param>
              </list>
            </param>
            <param name="searchFieldsToDisplay">
              <list>
                <param name="label">username</param>
                <param name="value">username</param>
              </list>
            </param>
            <module name="ConvertToIntention">
              <param name="settingToConvert">username</param>
              <param name="intention">
                <param name="name">addterm</param>
                <param name="arg">
                  <param name="username">$target$</param>
                </param>
                  <!-- tells the addterm intention to put our term in the first search clause no matter what. -->
                  <param name="flags"><list>indexed</list></param>
              </param>

              <!-- Time picker. -->
              <module name="TimeRangePicker">
                <param name="label">Time Picker</param>
                <param name="selected">Last 4 hours</param>
                <param name="searchWhenChanged">True</param>
                <module name="SubmitButton">
                  <param name="allowSoftSubmit">True</param>

                  <!-- Google Map. -->
                  <module name="GenericHeader" layoutPanel="graphArea">
                    <param name="label">Google Map</param>
                  </module>
                  <module name="HiddenSearch" layoutPanel="graphArea" autoRun="true">
                      <param name="search"> | dedup vpn_index | localop | geoip public_ip</param>
                      <module name="GoogleMaps">
                        <param name="drilldown">true</param>
                        <param name="drilldown_field">public_ip</param>
                        <param name="doubleClickZoom">off</param>
                        <param name="height">600px</param>
                        <param name="mapType">roadmap</param>
                        <param name="scrollwheel">off</param>
                        <param name="streetViewControl">on</param>

                        <!-- Search results in a table that updates with clicks. -->
                        <module name="HiddenSearch" autoRun="False">
                          <param name="search">index=vpn_access $clientips$</param>
                          <module name="ConvertToIntention">
                            <param name="settingToConvert">maps.drilldown</param>
                            <param name="intention">
                              <param name="name">stringreplace</param>
                              <param name="arg">
                                <param name="clientips">
                                  <param name="value">$target$</param>
                                </param>
                              </param>
                            </param>
                            <module name="JobProgressIndicator">
                            </module>
                            <module name="HiddenPostProcess">
                              <param name="search">dedup vpn_index | localop | geoip public_ip | rename public_ip as "public ip" assigned_ip as "assigned ip" public_ip_country_code as code public_ip_country_name as country public_ip_city as city public_ip_region_name as "state / region" bytes_rx as "bytes rx" bytes_tx as "bytes tx" | table username "public ip" "assigned ip" country code city "state / region" "bytes rx" "bytes tx" duration | sort - bytes_tx</param>
                              <module name="SimpleResultsTable">
                                <param name="drilldown">row</param>
                                <param name="count">1000</param>
                                <param name="entityName">results</param>
                                <module name="HiddenSearch">
                                  <param name="search">eventtype=public_ip="$public_ip$"</param>
                                  <module name="ConvertToIntention">
                                    <param name="settingToConvert">click.value</param>
                                    <param name="intention">
                                      <param name="name">stringreplace</param>
                                      <param name="arg">
                                        <param name="public_ip">
                                            <param name="value">$target$</param>
                                        </param>
                                      </param>
                                    </param>
                                    <module name="ViewRedirector">
                                      <param name="viewTarget">flashtimeline</param>
                                      <param name="popup">true</param>
                                    </module>
                                  </module>
                                </module>
                              </module>
                            </module>
                            <module name="ViewRedirectorLink">
                                <param name="label">View events...</param>
                                <param name="viewTarget">flashtimeline</param>
                            </module>
                        </module>
                      </module>

                    </module>
                  </module>
                </module>
              </module>
            </module>
          </module>
        </module>
      </module>
    </module>
  </module>
</module>

[hexx · Accepted solution]

I am counting 3 pull-down menus populated by cascading searches :

• 1st pull-down :

  index=vpn_access element_name=* | stats count by element_name | sort element_name

• 2nd pull-down :

  index=vpn_access $element_name$ | stats count by group_policy | sort group_policy

• 3rd pull-down :

  index=vpn_access $element_name$ $group_policy$ | stats count by username | sort username

As the size of your vpn_access index grows, the drop-down population will become more and more costly, as these searches are currently rigged to iterate through all events of that index.

I have one of two recommendations :

• Use the earliest and latest parameters of the SearchSelectLister module to restrict the time-range of the pull-down populating searches. See the module reference for SearchSelectLister for more information.

• If you absolutely need for the pull-down populating searches to iterate through all of the information contained in the vpn_access index, the best way to go is to set up a summary index from which these searches can generate the pull-down values. For more information on how to achieve this, see this topic in the Knowledge Manager manual.