feat(loops): filter and order data-row loops by a cell value - #348
Merged
DavidBabinec merged 10 commits intoAug 18, 2026
Conversation
A loop could pick a table and an order but not WHICH rows, so a section that should list the three featured articles listed the three newest ones. Migrated Webflow sites hit this immediately: their lists are curated by a boolean field the loop had no way to read. - New `@core/loops/cellFilter`: parse one condition out of the loop's filter bag and render it as SQL. Six operators (is / is not / checked / unchecked / has any value / empty), a closed set — never interpolated. - Both query paths apply it (post-type version join and data-kind direct read) and so do their COUNT queries, or pagination advertises rows the page query drops. - The canvas preview endpoint takes the same condition, so the editor shows what the published page will emit. - Properties panel renders a field picker from the selected table, and hides the value box for the operators that ignore it. Dialect notes, both learned the hard way and now pinned by tests: SQLite binds `?` by position in the TEXT, so the condition's parameters sit between tableId and limit/offset; and its json_extract returns INTEGER 1/0 for booleans, which never equals '1' across storage classes — hence the cast. The JSON read appears exactly once per fragment so the field name binds once, with coalesce folding in rows that lack the field. The field NAME binds as a parameter like the value, so no part of a filter reaches the statement text. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Ordering was limited to the row's own SQL columns, so a list could not be sorted by a real date, title, or rank that lives in cells_json. A migrated site could only approximate 'newest first' with import arrival order, which drifts the moment anything is re-imported. - `orderBy` now also accepts `cell:<fieldId>`; `parseCellOrder` reads it and `cellOrderSql` renders the expression with the field name bound as a parameter, so nothing reaches the SQL text. - Both query paths order by it, values compare as TEXT in both dialects (ISO dates sort chronologically; documented that numbers sort lexicographically — one predictable rule beats two engine-specific ones), and coalesce gives rows lacking the field a defined position. - The Loop panel lists the selected table's fields as order options. Riding on `orderBy` rather than a new prop means every caller that already threads it — publisher, canvas preview, imported data-order-by attributes — supports this without further plumbing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
"has any value" described the same condition in more words. The operator id was already `isSet`; the label now says so too. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ution CoreBunch#300 landed after this branch was cut and rewrote the same function. It widened loop media resolution from the built-in featured cell to every schema-declared media field, which meant reading `fields_json` next to `kind` and threading the field list into both row projections. The two changes never touch the same concern: this branch adds an optional cell condition to the count and page queries and a `cell:<field>` sort, and CoreBunch#300 changes which media ids get resolved afterwards. The resolution keeps both verbatim — `table.kind` and `collectMediaIds(rows, fields)` from upstream, the filter and order plumbing from here. Verified: tsc clean, and both loop suites green with no assertion failures (the two reported failures are Windows temp-file locking in test teardown, which reproduce on an untouched upstream checkout). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Merging upstream put this branch's filter/order plumbing and CoreBunch#300's wider media resolution into the same file, and the result crossed the 700-line module ceiling at 711 — CI caught it. Raising the ceiling would have been the wrong repair. The two halves answer different questions: the rest of the file decides WHICH rows a loop returns — the filter, the order, the page window — while this block decides what a row's media cells CONTAIN once those rows are in hand. They only ever met because both needed the same page of rows. So `resolveMediaIdsToPaths`, `collectMediaIds` and `resolvedMediaOverlay` move to `dataRowsMedia.ts` unchanged, and the source file imports them. No behaviour changes: 596 lines and 132 rather than 711. `MediaAssetRow` and the `readMediaCellIds` / `readRepeaterCell` imports went with them, and the one test that reached for `resolveMediaIdsToPaths` now imports it from where it lives. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… an id A relation cell stores the referenced row's ID. With `cellValue` rendered as a free-text box, filtering "Category is Impact" meant the author had to type `z6W88XshljjzArfa4sV0f` — a string the editor never shows anywhere and gives them no way to look up. The control was present and unusable, which is worse than absent: it looks like the feature works. When the chosen field is a relation, the referenced table's rows now load and `cellValue` becomes a select of their names. Everything else keeps the text box, and the valueless operators still drop the control entirely. Rows are labelled by their `name` cell with the slug as fallback — the same identity the Data workspace shows in its own row list, so the option an author picks here reads the same as the row they know. The fetch follows the pattern already in this component: lazy, keyed on the resolved target table, and a failed load degrades to an empty list rather than blocking the panel. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three ways the filter controls disagreed with the query behind them. Switching the loop's table left `cellField` pointing at the old table. The field picker falls back to its first option when the stored value isn't among its own, so the panel read "no filter" while the query still filtered on a column the new table doesn't have: an empty list with nothing on screen explaining it. Changing the table now clears the cell filter, and any `cell:` sort with it. The field picker offered every field, including ones a single condition can never match. A `multiSelect`, a `repeater`, or a multi-value `relation` stores a collection, and the SQL reads one cell as text, so the comparison runs against `["cat_impact"]` and no value the author picks can equal it. Media ids and page-tree refs are opaque strings the editor never shows. `isCellComparableField` keeps all of them out of both the filter and the order-by list, which is the same reasoning the relation value picker was added for. And the labels described the implementation rather than the task: "Only rows where" reading into "every row", plus "is set" and "is empty", which name the operator ids. Now "Filter by" / "No filter", "has any value" / "has no value", and cell sorts read "Field: Role" so they don't collide with the row's own columns. `cellFilterMatches` goes with them. It was exported for "the canvas preview and any future in-memory path", but the canvas fetches through the loop-preview endpoint and nothing else ever called it.
The suite seeded `cells_json` as `${JSON.stringify(cells)}`, which is
the one place in the repo that pre-stringifies that column; its sibling
`dataRowsFetch.test.ts` passes the object. On SQLite the column is TEXT
so both work, but Postgres stores it as jsonb and a pre-encoded string
lands as a jsonb *string* rather than an object. Every read then returns
null and 12 of the 16 assertions fail, whatever the feature does.
So the half of this change that Postgres actually runs, the
`#>> array[$n]` reads and the parameter numbering around them, had never
been executed. Seeding the way production writes fixes that: 16 green on
SQLite, 16 green against a real Postgres 16 via `DB=postgres
TEST_POSTGRES_URL=... bun test`. `system` binds as a boolean for the
same reason.
`loop-source-sql-safety.test.ts` widens to the whole `src/core/loops/`
tree. It exists to catch Postgres-isms in loop SQL and stopped at
`sources/`, which left `cellFilter.ts` one level up, the file carrying
the most engine-specific SQL in the subsystem, outside the gate.
The `data.rows` section still claimed filters "narrow by status, author, category-like fields, date", which was never true and is now wrong in a new way. It gets the real contract instead: the three `filters` keys, the closed operator set, the `cell:<fieldId>` order form, which fields either picker offers and why the rest are excluded, and the rule that changing the table clears both. The canvas-path table gains the loop-preview endpoint's new query params, and the walkthrough now names the controls an author sees.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Lets a
base.loopon thedata.rowssource pick which rows it lists, and which cell it sorts by:is,is not,is checked,is unchecked,has any value,has no value.Field: <Label>, alongside the existing row columns.Both are configured in the loop's properties panel and take effect in the canvas preview, in published pages, and in the "load more" endpoint.
Why
A loop could pick a table, an order, and a limit, but not a subset. So a page meant to show the four team members flagged for the About page showed the four most recently updated rows instead, and a list meant to follow the source CMS's own publish date followed our
updated_atcolumn. The only workarounds were reordering rows by hand or duplicating a table, and both drift the moment someone edits an item.This is deliberately one condition rather than a query builder: it covers the real cases (a "featured" checkbox, a category, a "show on homepage" flag) without inventing an AND/OR grammar the editor can't express and future maintainers would have to keep sound in two dialects.
How
New pure module
src/core/loops/cellFilter.tsowns the whole contract: parsing, the closed operator set, the SQL fragments, and the rule for which fields a single condition can address.cells_json #>> array[$n], SQLitejson_extract(cells_json, '$.' || ?). A hostile field id cannot reach the statement (covered by a test).coalesce(…, '')folds the missing-field case into the comparison instead of needing a secondis nullbranch; repeating the expression would repeat its placeholder while the caller binds the field once.json_extractreturns INTEGER1/0for JSON booleans, and SQLite compares by storage class first, so1 = '1'is false without the cast. The checked/unchecked operators accept both spellings.orderByascell:<fieldId>rather than a new prop, so every caller that already threadsorderBy(publisher, canvas preview endpoint, importeddata-order-by) gets it for free. Values compare as text: ISO dates sort chronologically, which is the case this exists for.COUNTapplies the same filter on both the post-type and data-kind paths, so pagination never advertises rows the page query drops.Touched:
loops/cellFilter.ts(new),loops/sources/dataRows.ts,LoopPropertiesView.tsx,useLoopPreviewItems.ts,server/handlers/cms/data/tables.ts,core/persistence/cmsData.ts.User impact
Additive. A loop with no cell field configured behaves exactly as before, and a half-configured filter (field picked, operator not yet) keeps listing everything rather than silently emptying the list.
Verification
Two new files:
src/__tests__/loops/cellFilter.test.ts(parsing, parameter binding, dialect shapes, which fields a condition may address) andsrc/__tests__/loops/dataRowsCellFilter.test.ts(behaviour against a real migrated SQLite database, both table kinds, filtered counts, pagination).Also exercised end-to-end on a real site migrated from another CMS: an About page list that needed 4 of 30 rows, and an index page that had to follow the source CMS's publish date rather than ours.
Review changes
Three defects found in review, fixed on this branch:
cellFieldnaming a column the new table does not have. The field picker falls back to its first option when the stored value is not among its own, so the panel read "no filter" while the query still filtered and returned nothing. Changing the table now clears the cell filter and anycell:sort.multiSelect,repeater, or multi-valuerelationstores a collection; the SQL reads one cell as text, so the comparison ran against["cat_impact"]and no picked value could match.isCellComparableFieldkeeps those, plus media ids and page-tree refs, out of both pickers.cells_jsonpre-stringified, which lands as a jsonb string rather than an object, so 12 of 16 assertions failed on Postgres whatever the feature did. Seeding the way production writes fixes that, and the#>> array[$n]half is now covered: 16 green on SQLite, 16 green against a real Postgres 16.Also: clearer control labels (Filter by / No filter,
has any value/has no value,Field: <name>sorts),loop-source-sql-safety.test.tswidened to the wholesrc/core/loops/tree so the gate coverscellFilter.ts, the unusedcellFilterMatchesexport removed, and aCHANGELOG.mdentry plusdocs/features/loops.mdbrought up to date.