All Apps and Add-ons

searches running serially

rizzo75
Path Finder

I have a Sideview Utils created view/dashboard that requires many searches and I want them to run serially. The modules are organized like this:


root
|
TimeRangePicker
|
Button
| | | | | |
Search Search Search Search Search Search
... ... ... ... ... ...

I am at a loss on how do this. The only solution I can think of is to chain the searches together with the Gate module, but that does not seem ideal.

Thanks,
Joe

sideview
SplunkTrust
SplunkTrust

Sorry for the delay responding.

I don't think that simply waiting until the http POST for the first has returned with a search id, will help.

If your end goal is to try and avoid hitting the max number of concurrent searches, you may have some other options and you can read more here - http://answers.splunk.com/answers/82/i-keep-getting-this-max-concurrent-searches-reached-error-what-...

But waiting for the search id's return wont buy you enough time to avoid significant search overlap; the search id's come back right away and it means the job has just started, not that it's anywhere near finishing.

Since the server is a VM can you simply allocate more cores to that VM? This may be a quick way to solve this problem since the concurrent search ceiling is set by the number of cores.

And lastly, the answer to your actual question. 😃

There is a way to make searches run truly in series without using any custom code, but it uses both Gate module and something pretty arcane in ValueSetter, and I frankly wouldn't go that way. What I recommend instead is an approach that uses Gate modules with a CustomBehavior. Although the CustomBehavior means there's a little custom Javascript involved, mentally this approach is much simpler and that makes it on the whole better.

here's what the hierarchy would look like.

                             root
                  |                     |              |
            TimeRangePicker           Gate           Gate
                  |                     |              |
                Button               Search         Search
                  |                    ...            ...
                Search 
         |                 |          |
JobProgressIndicator     Pager     CustomBehavior
                          |
                        Table

except that underneath those two other Gate+Search branches, you'd have another
CustomBehavior + Gate hanging off of each.

All these CustomBehavior modules would have <param name="customBehavior">waitForJobDone</param> and <param name="requiresDispatch">True</param>

But better yet, here's an actual working XML view. This sets up three searches to dispatch serially, in that the second search doesn't dispatch until the first one is completely finished, etc.. It uses only splunkd's internal data so you can run this on your own Splunk instance as a local example.

<view autoCancelInterval="90" isVisible="false" onunloadCancelJobs="true" template="dashboard.html" isSticky="False">
  <label>Testcase for sequential dispatch via CustomBehavior and Gate</label>
  <module name="AccountBar" layoutPanel="appHeader" />
  <module name="AppBar" layoutPanel="appHeader" />
  <module name="SideviewUtils" layoutPanel="appHeader" />

  <module name="Message" layoutPanel="messaging">
    <param name="filter">*</param>
    <param name="maxSize">2</param>
    <param name="clearOnJobDispatch">False</param>
  </module>

  <module name="TimeRangePicker" layoutPanel="panel_row1_col1" autoRun="True">
    <param name="default">Last 30 days</param>

    <module name="Button">

      <module name="Search">
        <param name="search">index=_internal source=*metrics.log group="per_sourcetype_thruput" | stats sum(kb) by series</param>

        <module name="HTML">
          <param name="html"><![CDATA[
          <h3>$results.count$ sourcetypes found $search.timeRange.label$</h3>
          ]]></param>
        </module>

        <module name="CustomBehavior">
          <param name="customBehavior">waitForJobDone</param>
          <module name="Gate">
            <param name="to">gate2</param>
          </module>
        </module>

        <module name="JobProgressIndicator"></module>

        <module name="Pager">
          <module name="SimpleResultsTable"></module>
        </module>
      </module>
    </module>
  </module>

  <module name="Gate" layoutPanel="panel_row2_col1">
    <param name="id">gate2</param>

    <module name="Search">
      <param name="search">index=_internal source=*metrics.log group="per_sourcetype_thruput" | stats sum(kb) by series</param>

      <module name="CustomBehavior">
          <param name="customBehavior">waitForJobDone</param>
          <module name="Gate">
            <param name="to">gate3</param>
          </module>
        </module>


      <module name="JobProgressIndicator"></module>

      <module name="Pager">
        <module name="SimpleResultsTable"></module>
      </module>
    </module>

  </module>

  <module name="Gate" layoutPanel="panel_row3_col1">
    <param name="id">gate3</param>

    <module name="Search">
      <param name="search">index=_internal source=*metrics.log group="per_sourcetype_thruput" | stats sum(kb) by series</param>
      <module name="JobProgressIndicator"></module>

      <module name="Pager">
        <module name="SimpleResultsTable"></module>
      </module>
    </module>
  </module>

</view>

and last but not least, here is the customBehavior definition you would have to put inside your application.js, or in a custom JS file using the customJavascript param. Note that if you don't have this customBehavior in application.js the above example view will not work.

if (typeof(Sideview)!="undefined") {
    Sideview.utils.declareCustomBehavior("waitForJobDone", function(customBehaviorModule) {
        var methodReference = customBehaviorModule.isReadyForContextPush.bind(customBehaviorModule);

        customBehaviorModule.isReadyForContextPush = function() {
            var context = this.getContext();
            var search  = context.get("search")
            if (!search.job || !search.job.isDone()) {
                this.deferring = true;
                return Splunk.Module.DEFER;
            }
            return methodReference();
        }

        customBehaviorModule.onJobDone = function() {
            if (this.deferring) {
                this.pushContextToChildren();
                this.deferring = 0;
            }
        }
    });
}

Also note that the first time you create an application.js file in a given app, you have to restart splunkweb for the file to be picked up. And note that you'll need at least Sideview Utils 2.5 to get the Gate module.

0 Karma

rizzo75
Path Finder

can you elaborate a little on the end goal? to lessen the performance impact on the Splunk server (happens to be a VM)
To put less of a burden on Splunkd by running them one at a time? yes
Also by serially do you mean that the second one shouldn't be dispatched until the first one completes? yes - however any solution to have many searches supporting one view without impacting performance will work.
Or that the second one shouldn't be dispatched until the http POST for the first has returned with a search id? I suspect this would work too.

Thanks,
Joe

0 Karma

sideview
SplunkTrust
SplunkTrust

Some questions - can you elaborate a little on the end goal? To put less of a burden on Splunkd by running them one at a time? Also by serially do you mean that the second one shouldn't be dispatched until the first one completes? Or that the second one shouldn't be dispatched until the http POST for the first has returned with a search id? If you can update your answer or comment back, then I'll write up a real answer.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...