All Apps and Add-ons

Nesting PostProcess modules

tiberious726
Path Finder

Say I want my view to produce a search of the form A|B|C, is it possible to use the post process module in this way?

search---A
    postprocess --- B
        postprocess --- C

Or does splunk insist on associating a given postprocess solely with its base search?

1 Solution

sideview
SplunkTrust
SplunkTrust

Well, if the postProcesses are all operating on the same dispatched search, then you can in each case refer to the aggregate postProcess from upstream as $postProcess$, pretty much anywhere.

<module name="PostProcess">
  <param name="search">| stats count by username, host, error_code</param>

  ...
  <module name="PostProcess">
    <param name="search">$postProcess$ | stats sum(count) as count by username</param>

    ...
    <module name="PostProcess">
      <param name="search">$postProcess$ | where count>10 | sort - count | head 10</param>

And this will work very well.

On the other hand, if one or more of the PostProcess modules is going to be operating on a different dispatched search, then be aware that as of Sideview Utils 2.2.7, the postProcess search will get automatically cleared when a given dispatched search B supercedes search A in the data being passed downstream. In other words you'd find that you were unable to reference the $postProcess$ from below that transition. that said, you can always use the ValueSetter module to temporarily clone any context key to any other key, and this can hide it from being nulled out.

<module name="ValueSetter">
  <param name="name">secretPostProcess</param>
  <param name="value">$postProcess$</param>

It's a weird and silly trick, but sometimes useful in strange cases, like when two different UI elements need to write to exactly the same key name like "username", but at a point deeper in the page you need to access both of the values together.

View solution in original post

sideview
SplunkTrust
SplunkTrust

Well, if the postProcesses are all operating on the same dispatched search, then you can in each case refer to the aggregate postProcess from upstream as $postProcess$, pretty much anywhere.

<module name="PostProcess">
  <param name="search">| stats count by username, host, error_code</param>

  ...
  <module name="PostProcess">
    <param name="search">$postProcess$ | stats sum(count) as count by username</param>

    ...
    <module name="PostProcess">
      <param name="search">$postProcess$ | where count>10 | sort - count | head 10</param>

And this will work very well.

On the other hand, if one or more of the PostProcess modules is going to be operating on a different dispatched search, then be aware that as of Sideview Utils 2.2.7, the postProcess search will get automatically cleared when a given dispatched search B supercedes search A in the data being passed downstream. In other words you'd find that you were unable to reference the $postProcess$ from below that transition. that said, you can always use the ValueSetter module to temporarily clone any context key to any other key, and this can hide it from being nulled out.

<module name="ValueSetter">
  <param name="name">secretPostProcess</param>
  <param name="value">$postProcess$</param>

It's a weird and silly trick, but sometimes useful in strange cases, like when two different UI elements need to write to exactly the same key name like "username", but at a point deeper in the page you need to access both of the values together.

guilhem
Contributor

Indeed, it's simplier than my workaround, append postprocesses within the same postprocess using $postProcess$ | "your new postprocess", no need to set a value setter here.

0 Karma

guilhem
Contributor

First, before sideview utils 2.2.7? the content of a postprocess module was kept until another postprocess module replace its content. As of 2.2.7, it is no longer the case, as the content of the search parameter of the postprocess module is reseted after use (see topics about the previous behavior).

In any case, whatever version you are using, you cannot intricate postprocess like that, as the search parameter of the postprocess module will not append one after the other but rather get replaced by the new parameter set downstream.

To be convinced, I have attached a simple test case that illustrate this:

sample view

I really recommend that you try to open this view with sideview editor and select runtime Debug, then click on the modules you want to see, in order to understand how params are passed downstream.

A workaround would be to pass the content of the first postprocess module inside a variable (using a value setter module for example), and then use this value in the second postprocess:

SEARCH
| POSTPROCESS 1
| VALUESETTER $mypostprocess$ = $postprocess$
| POSPROCESS 2 search= $mypostprocess$ | "your second postprocess"

(I haven't tried it so this may not work)

More generally why do you need to nest 2 postprocess? Why not put the two postprocess inside the same one, and then filter downstream on the results you need?

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...