All Apps and Add-ons

Nesting PostProcess modules

tiberious726
Path Finder

Say I want my view to produce a search of the form A|B|C, is it possible to use the post process module in this way?

search---A
    postprocess --- B
        postprocess --- C

Or does splunk insist on associating a given postprocess solely with its base search?

1 Solution

sideview
SplunkTrust
SplunkTrust

Well, if the postProcesses are all operating on the same dispatched search, then you can in each case refer to the aggregate postProcess from upstream as $postProcess$, pretty much anywhere.

<module name="PostProcess">
  <param name="search">| stats count by username, host, error_code</param>

  ...
  <module name="PostProcess">
    <param name="search">$postProcess$ | stats sum(count) as count by username</param>

    ...
    <module name="PostProcess">
      <param name="search">$postProcess$ | where count>10 | sort - count | head 10</param>

And this will work very well.

On the other hand, if one or more of the PostProcess modules is going to be operating on a different dispatched search, then be aware that as of Sideview Utils 2.2.7, the postProcess search will get automatically cleared when a given dispatched search B supercedes search A in the data being passed downstream. In other words you'd find that you were unable to reference the $postProcess$ from below that transition. that said, you can always use the ValueSetter module to temporarily clone any context key to any other key, and this can hide it from being nulled out.

<module name="ValueSetter">
  <param name="name">secretPostProcess</param>
  <param name="value">$postProcess$</param>

It's a weird and silly trick, but sometimes useful in strange cases, like when two different UI elements need to write to exactly the same key name like "username", but at a point deeper in the page you need to access both of the values together.

View solution in original post

sideview
SplunkTrust
SplunkTrust

Well, if the postProcesses are all operating on the same dispatched search, then you can in each case refer to the aggregate postProcess from upstream as $postProcess$, pretty much anywhere.

<module name="PostProcess">
  <param name="search">| stats count by username, host, error_code</param>

  ...
  <module name="PostProcess">
    <param name="search">$postProcess$ | stats sum(count) as count by username</param>

    ...
    <module name="PostProcess">
      <param name="search">$postProcess$ | where count>10 | sort - count | head 10</param>

And this will work very well.

On the other hand, if one or more of the PostProcess modules is going to be operating on a different dispatched search, then be aware that as of Sideview Utils 2.2.7, the postProcess search will get automatically cleared when a given dispatched search B supercedes search A in the data being passed downstream. In other words you'd find that you were unable to reference the $postProcess$ from below that transition. that said, you can always use the ValueSetter module to temporarily clone any context key to any other key, and this can hide it from being nulled out.

<module name="ValueSetter">
  <param name="name">secretPostProcess</param>
  <param name="value">$postProcess$</param>

It's a weird and silly trick, but sometimes useful in strange cases, like when two different UI elements need to write to exactly the same key name like "username", but at a point deeper in the page you need to access both of the values together.

guilhem
Contributor

Indeed, it's simplier than my workaround, append postprocesses within the same postprocess using $postProcess$ | "your new postprocess", no need to set a value setter here.

0 Karma

guilhem
Contributor

First, before sideview utils 2.2.7? the content of a postprocess module was kept until another postprocess module replace its content. As of 2.2.7, it is no longer the case, as the content of the search parameter of the postprocess module is reseted after use (see topics about the previous behavior).

In any case, whatever version you are using, you cannot intricate postprocess like that, as the search parameter of the postprocess module will not append one after the other but rather get replaced by the new parameter set downstream.

To be convinced, I have attached a simple test case that illustrate this:

sample view

I really recommend that you try to open this view with sideview editor and select runtime Debug, then click on the modules you want to see, in order to understand how params are passed downstream.

A workaround would be to pass the content of the first postprocess module inside a variable (using a value setter module for example), and then use this value in the second postprocess:

SEARCH
| POSTPROCESS 1
| VALUESETTER $mypostprocess$ = $postprocess$
| POSPROCESS 2 search= $mypostprocess$ | "your second postprocess"

(I haven't tried it so this may not work)

More generally why do you need to nest 2 postprocess? Why not put the two postprocess inside the same one, and then filter downstream on the results you need?

Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...