Hi all,
I am having difficulty working around what I understand to be a hard limit of 10000 events for the HiddenPostProcess module. I am looking for a way to reduce the granularity of the events going across, but without success.
Example problem code:
<module name="HiddenSearch" autoRun="False" layoutPanel="mainSearchControls">
<param name="search">index=analysisindex01
eventtype="EVENTS01" OR
eventtype="EVENTS02" OR
eventtype="EVENTS03" OR
eventtype="EVENTS04" OR
eventtype="EVENTS05" OR
eventtype="EVENTS06" OR
eventtype="EVENTS07" OR
eventtype="EVENTS08"
timeformat="%d/%m/%Y"
starttime=10/11/2010
searchtimespandays=5
| fields eventtype, _time
</param>
... stuff ...
<module name="HiddenPostProcess" layoutPanel="panel_row2_col1">
<param name="search">timechart span="1d" count(eventtype) by eventtype</parm>
This will fail. It will produce results, but they will be capped to 10000.
If I do
timeformat="%d/%m/%Y"
starttime=10/11/2010
searchtimespandays=5
| bucket _time span=1h
| fields eventtype, _time
in an attempt to summarize the events a bit, I only get 1 row, for 1 day, in the reports generated, and the numbers in it make no sense.
Is there something I am missing?
Any and all help is greatly appreciated.
bucket doesn't actually summarize at all. bucket just discretizes the value, e.g. in your search, it means for each event, snap the value of _time to the previous hour boundary.
What you'd want to do is something like
timeformat="%d/%m/%Y" starttime=10/11/2010 searchtimespandays=5 | bucket _time span=1h | stats count by _time eventtype
That will give you 1 row per day per eventtype
then in your hidden post process, you can do
timechart span=1d sum(count) by eventtype
bucket doesn't actually summarize at all. bucket just discretizes the value, e.g. in your search, it means for each event, snap the value of _time to the previous hour boundary.
What you'd want to do is something like
timeformat="%d/%m/%Y" starttime=10/11/2010 searchtimespandays=5 | bucket _time span=1h | stats count by _time eventtype
That will give you 1 row per day per eventtype
then in your hidden post process, you can do
timechart span=1d sum(count) by eventtype