I have a query that has become quite complex and now takes several minutes to run. It seemed a perfect candidate for accelerating with a scheduled search and a summary index.
The query itself uses an evaluated field to distinguish events in two overlapping time periods.
When the query is run "normally" (ie. not populating the summary index) all works fine.
However, when the query runs to populate the summary index, via the sistats command, the evaluated field is stored as a reserved field: psrsvd_ct_eval(T==15) and psrsvd_ct_eval(T==60).
In the original query these evaluated fields were counted as 15 & 60 minute counts. However, when the stats command is run against the populated summary index, the values come back as zero.
If I explicitly try to include the psrsvd_ct_* fields in my stats report, I get an error saying they are for internal use. I'm so close to what I need - the right data is in the index - it's just wrongly named and non-reportable.
I've spent hours reading Splunk docs and googling this - there seem to be a few "workarounds" but nothing that solves my problem.
Does anyone know how to resolve this?
The (much simplified) basic summary index populating search looks like this:
sourcetype=XX earliest=-1500m latest=-1440m location=* | where(isnotnull(error_id)) | eval T=1 | append [similar search for another period | eval T=2] | sistats count(eval(T==1)) as count1, count(eval(T==2)) as count2 by location
Then, reporting on that, I use:
search index=summary search_name="Summary Count" | stats count(eval(T==1)) as count1, count(eval(T==2)) as count2 by location
However, count1 and count2 are always reported as 0, even though the correct data is present in the form of fields with the name/value: psrsvd_ct_eval(T==1)=num1 psrsvd_ct_eval(T==2)=num2.
