In some conditions the head command knows that the search has completed all the information that the user asked for, and it reaches back into the search pipeline and shuts down the search.
EG: if you run
index=_internal
over all time, it'll take a really long time. But if you run
index=_internal | dedup group | head 5
it'll complete in a few seconds.
To take another example
index=_internal | stats count by group | head 5
it's pretty similar, but the system knows that the counts are still going to increase, so it lets this search run to completion.
Is there a good summary in the docs or in search.bnf that explain under what circumstances we can rely on this behavior?
No, the specific conditions that head
will preemptively stop search aren't enumerated. In general it will halt search when the results remain correct if search is preempted. This means that any search that is fully distributable or that can incrementally process events once sorted by time will preempt with head
. If any search command before head
requires seeing every event to emit a final result, say a transforming command like stats or a command like sort, the search cannot be preempted for correctness.
No, the specific conditions that head
will preemptively stop search aren't enumerated. In general it will halt search when the results remain correct if search is preempted. This means that any search that is fully distributable or that can incrementally process events once sorted by time will preempt with head
. If any search command before head
requires seeing every event to emit a final result, say a transforming command like stats or a command like sort, the search cannot be preempted for correctness.