How to include events with non existing fields in a search string ?

[AmineTN]

After fixing filters on some fields that don't exist in all the events, I tried to apply these filters on the graphs and the problem here is that when Splunk reads the search string of a graph, it gets only the events where the fields exist and it excludes the other events. As a result all the statistics and the graphs are wrong !!

Ayone has a solution please ??

Thanks in Advance.

[gcusello]

Hi @AmineTN ,

good for you, see next time!

Ciao and happy splunking

Giuseppe

P.S.: Karma Points are appreciated by all the contributors 😉

[ITWhisperer]

It is not clear to me what it is you are trying to do - can you give an example of the search you are doing and where the filters are defined and used? If the statistics are incorrect, are the filters being applied at the correct point in the search?

[AmineTN]

I will explain more here with some examples.

For these two JSON events the first contain the field "SuperForceCouponStatus" but the second don't.

Then I made a filter for "SuperForceCouponStatus" as following :

<input type="dropdown" token="filter_SuperForce">
<label>SuperForce</label>
<choice value="*">ALL</choice>
<prefix>SuperForceCouponStatus="</prefix>
<suffix>"</suffix>
<fieldForLabel>SuperForceCouponStatus</fieldForLabel>
<fieldForValue>SuperForceCouponStatus</fieldForValue>
<search>
<query>|tstats count where index=$filter_index$ $filter_backend$ 
by SuperForceCouponStatus</query>
<earliest></earliest>
<latest>now</latest>
</search>
<default>*</default>
</input>

 After that I made a graph to display the number of events depending on the filters qs following :

<panel>
<single>
<title>Total number of Transactions</title>
<search>
<query>| tstats count where index=$filter_index$
$filter_backend$
$filter_status$
$filter_SuperForce$ |</query>
<earliest>$time_range.earliest$</earliest>
<latest>$time_range.latest$</latest>
</search>
<option name="colorBy">value</option>
<option name="colorMode">block</option>
<option name="drilldown">none</option>
<option name="numberPrecision">0</option>
<option name="rangeColors">["0x53a051","0x0877a6","0xf8be34","0xf1813f","0xdc4e41"]</option>
<option name="rangeValues">[100,10000,1000000,10000000]</option>
<option name="refresh.display">progressbar</option>
<option name="showSparkline">1</option>
<option name="showTrendIndicator">1</option>
<option name="trellis.enabled">0</option>
<option name="trellis.scales.shared">1</option>
<option name="trellis.size">medium</option>
<option name="trendColorInterpretation">standard</option>
<option name="trendDisplayMode">absolute</option>
<option name="underLabel">transactions</option>
<option name="unitPosition">after</option>
<option name="useColors">1</option>
<option name="useThousandSeparators">1</option>
</single>
</panel>

And the problem is that this graph takes only the events that have the field "SuperForceCouponStatus" because $filter_SuperForce$ is applied in its search string !!

Hope the problem is more clear now 🙂

[ITWhisperer · Accepted solution]

So, essentially, your ALL option is filtering out all stats events with null SuperForceCouponStatus. Try changing the value of the drop down options to include the condition or an empty string when ALL is selected. Something like this

<input type="dropdown" token="filter_SuperForce">
<label>SuperForce</label>
<choice value="">ALL</choice>
<fieldForLabel>SuperForceCouponStatus</fieldForLabel>
<fieldForValue>SuperForceCouponStatusFilter</fieldForValue>
<search>
<query>|tstats count where index=$filter_index$ $filter_backend$ 
by SuperForceCouponStatus
| eval SuperForceCouponStatusFilter="SuperForceCouponStatus=\"".SuperForceCouponStatus."\""</query>
<earliest></earliest>
<latest>now</latest>
</search>
<default></default>
</input>

[AmineTN]

Thanks a lot @ITWhisperer , It's working perfectly !!!

[gcusello]

Hi @AmineTN,

did yu tried to add a fillnull to give a value (e.g. "N.A. " or "-") to this field when there isn't any value?

| fillnull value="N.A." SuperForceCouponStatus

Ciao.

Giuseppe

[AmineTN]

Yes I tried and the problem the fillnull command is applied after filtering the events, So it's useless !!

Thanks for your try ❤️