All Apps and Add-ons

Sideview Utils PostProcess and local csv inputlookup too slow!

alvaromari83
Path Finder

Hello all!

I'm implementing a search panel with 2 sideview pulldowns. First one is just made of 3 static options, that serve as arguments in the nested second pulldown module, which queries its values from a csv lookup input file, using a PostProcess module. The query for this inputlookup is:

| inputlookup file.csv | where fuente="source1" | fields nombre valor

This query is running fast as hell in the search app, as expected, as the csv itself is just a few rows with the following format:

fuente , nombre , valor
source1 , Matricula , cot_carplate
source1 , Nombre , cot_nombre
source1 , Documentos , cot_id
soruce2 , Numero pol , pol_pol

The view XML is the following:

<!-- First pulldown list, static values for sourcetypes -->

fuente
Fuente de datos:
left


Cotizaciones
source1


Polizas
source2


Siniestros
source3


Busqueda libre


<!-- Second pulldown list to select the search parameters to use. It depends on the first pulldown list -sourcetypes to search from-. Uses a static lookup csv table, which is searched in postProcess param using the previous pulldown result as argument -$fuente$-. The postprocess lookup returns labelnames and values for the search params valid for the selected sourcetype. In free text search, this part of the resultant seach query will be empty -->
<module name="Pulldown">
 <param name="float">left</param>
 <param name="name">valor</param>
 <param name="label">Parametro de busqueda:</param>

 <param name="postProcess">
    <![CDATA[
     | inputlookup file.csv | where fuente="$fuente$" | fields nombre valor 
    ]]>
 </param>
 <param name="staticOptions"/>

 <!-- Equal symbol to avoid issues in free text search (no sourcetype is specified in the first pulldown list). -->
 <param name="template">$value$ =</param>
 <param name="valueField">valor</param>
 <param name="labelField">nombre</param

When I load this view, the second pulldown population through postProcess takes a lot of time (almost 7-8 seconds). I'm on the last Sideview Utils version on Splunk 5.0. Any idea of what could be the issue?

Thanks and regards!

sideview
SplunkTrust
SplunkTrust

Yes I know what the problem is. A postprocess search always exists in relation to some base search and is meaningless without some base search.

Here you are using a postprocess search to fill the second Pulldown but there is no base search. Unfortunately there is always an implied base search of "*" over all time. So you see the problem. 😃 The dynamic Pulldown tells the ui framework "I require search results", and so the ui framework obligingly dispatches a search for it, with a dispatch point at the level of that Pulldown module. However the search dispatched is "*", over all time. (!!!) Which is bad because this search can take an extremely long time depending on how much data you have indexed.

It is of course easy to forget this when the postprocess search is itself a generating command like inputlookup.

The answer is simply to move your postprocess param into a search module:

<module name="Pulldown">
  <param name="name">fuente</param>
  ...

  <module name="Search">
    <param name="search">| inputlookup file.csv | where fuente="$fuente$" | fields nombre valor</param>

    <module name="Pulldown">
      <param name="name">valor</param>
      ...

and to not use the Pulldown module's postprocess param at all here.

0 Karma

alvaromari83
Path Finder

Allright, you nailed it! Your explanation absolutely makes sense, I will stream the pulldown after a search.

Thank you!

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...