under what situations will the head command intelligently finalize searches?

[sideview]

In some conditions the head command knows that the search has completed all the information that the user asked for, and it reaches back into the search pipeline and shuts down the search.

EG: if you run

index=_internal 

over all time, it'll take a really long time. But if you run

index=_internal | dedup group | head 5

it'll complete in a few seconds.

To take another example

index=_internal | stats count by group | head 5

it's pretty similar, but the system knows that the counts are still going to increase, so it lets this search run to completion.

Is there a good summary in the docs or in search.bnf that explain under what circumstances we can rely on this behavior?

[Stephen_Sorkin · Accepted solution]

No, the specific conditions that head will preemptively stop search aren't enumerated. In general it will halt search when the results remain correct if search is preempted. This means that any search that is fully distributable or that can incrementally process events once sorted by time will preempt with head. If any search command before head requires seeing every event to emit a final result, say a transforming command like stats or a command like sort, the search cannot be preempted for correctness.