Skip to content

Commit 6ef4046

Browse files
committed
Document that PPL fetch_size limits aggregation buckets
fetch_size applies to the final rows of the query, so on an aggregating query it limits the number of buckets returned rather than sampling documents. The existing docs described it only as a row limit, which reads as DSL size -- where aggregation results are returned in full. Contrast both DSL analogues: unlike terminate_after it does not corrupt aggregate values, and unlike size it does truncate aggregation results. Signed-off-by: Kai Huang <ahkcs@amazon.com>
1 parent 79d532d commit 6ef4046

2 files changed

Lines changed: 13 additions & 1 deletion

File tree

docs/user/interfaces/endpoint.rst

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -330,6 +330,18 @@ The ``fetch_size`` parameter limits the number of rows returned in a PPL query r
330330

331331
If ``fetch_size`` is larger than ``plugins.query.size_limit``, the result is capped at ``plugins.query.size_limit``. The effective number of rows returned is always ``min(fetch_size, plugins.query.size_limit)``.
332332

333+
Behavior with aggregations
334+
--------------------------
335+
336+
``fetch_size`` applies to the **final** rows of the query, equivalent to appending ``| head <fetch_size>``. For an aggregating query (``stats``, ``timechart``, ``top``, ``rare``, ``patterns ... mode=aggregation``, and similar) those rows are buckets, so ``fetch_size`` limits the number of **buckets** returned::
337+
338+
source=logs | stats count() by extension -> 6 buckets
339+
source=logs | stats count() by extension (fetch_size=3) -> first 3 buckets only
340+
341+
Aggregate values are unaffected — every matching document is still visited, so each returned bucket holds its full count. Unlike DSL ``terminate_after``, which undercounts, ``fetch_size`` drops whole rows off the end rather than corrupting values. But unlike DSL ``size``, which leaves aggregation results intact, it does truncate them.
342+
343+
Bucket counts grow with the queried time range, not the data volume: ``stats count() by span(@timestamp, 1h), extension`` over 7 days with 6 extensions yields ``168 x 6 = 1008`` rows. Use an explicit ``head`` when a bounded aggregation result is wanted.
344+
333345
Note
334346
----
335347

docs/user/ppl/limitations/limitations.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,7 @@ For the following functionalities, the query will be forwarded to the V2 query e
6262
* ML
6363
* Kmeans
6464
* `show datasources` and command
65-
* SQL queries with `fetch_size` parameter (cursor-based pagination). Note: PPL's `fetch_size` (response size limiting, no cursor) is supported in Calcite Engine.
65+
* SQL queries with `fetch_size` parameter (cursor-based pagination). Note: PPL's `fetch_size` (response size limiting, no cursor) is supported in Calcite Engine. It limits the final rows of the query, so on an aggregating query it limits the number of buckets returned.
6666

6767

6868
## Malformed Field Names in Object Fields

0 commit comments

Comments
 (0)