Skip to content

Commit d18eb2a

Browse files
committed
feat(cli): make EQL reinstall dependency-safe
1 parent 233349a commit d18eb2a

44 files changed

Lines changed: 3763 additions & 770 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.changeset/safe-eql-reinstall.md

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
---
2+
"stash": patch
3+
---
4+
5+
Preserve encrypted data and reconstruct functional indexes when reinstalling EQL v3, while refusing unsupported external dependencies before mutation.
6+
7+
`stash eql install` and `stash eql upgrade` now capture dependent functional
8+
indexes before replacing the disposable EQL schemas, then restore and verify
9+
their definitions, clustering, replica-identity role, comments, ownership,
10+
explicit statistics targets, and health in the same transaction. A
11+
reconstruction failure rolls the replacement back.
12+
Unsupported dependencies—including views, policies, constraints, and
13+
partitioned indexes—are named and refused before mutation.
14+
15+
Reinstall remains a maintenance-window operation: its advisory lock serializes
16+
`stash` lifecycle commands, not unrelated database DDL. Generated EQL migrations
17+
contain the raw bundle and do not include these reinstall protections.

.github/workflows/tests.yml

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -333,8 +333,10 @@ jobs:
333333
# (`installer/__tests__/verify.live.test.ts`) would run in no CI
334334
# workflow at all: a routine `@cipherstash/eql` bump could then make
335335
# every `stash eql install` fail with phantom damage, on green CI.
336-
# These suites need Postgres only, no CipherStash credentials; the
337-
# verify suite installs EQL v3 into its own schemas, which coexists
336+
# Most suites need Postgres only. The encrypted-index upgrade suite also
337+
# loads this job's CipherStash credentials from packages/stack/.env and
338+
# uses the binding built above to create genuine ciphertext. The verify
339+
# suite installs EQL v3 into its own schemas, which coexists
338340
# with the image's pre-installed EQL v2 that the stack tests use.
339341
# They share that one database, so the CLI vitest config runs them
340342
# serially (the `live` project sets `fileParallelism: false` —

AGENTS.md

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -84,10 +84,6 @@ If these variables are missing, tests that require live encryption will fail or
8484
- `packages/bench`: Performance / index-engagement benchmarks (private, not published)
8585
- `packages/protect-ffi`: Native FFI bindings to the CipherStash Client SDK (`@cipherstash/protect-ffi`) — the Rust core that `packages/stack` encrypts and decrypts through, absorbed from `cipherstash/protectjs-ffi`. Contains a **nested Cargo workspace** (`crates/`) and six per-platform binary packages under `platforms/*`, each published as `@cipherstash/protect-ffi-<platform>` and linked here via `workspace:*`. Also holds the repo's live FFI integration suite at `integration-tests/` — a private workspace member (`@cipherstash/ffi-integration-tests`) enrolled by its own literal entry in `pnpm-workspace.yaml`, needing Docker and credentials, and deliberately carrying **no `test` script** so `pnpm test` cannot reach it. See the "Working on protect-ffi" notes below before touching it — its default `test` and `build` are deliberately Rust-free.
8686
- `packages/eql`: The Encrypt Query Language subtree — the SQL bundle that stores and queries encrypted payloads — absorbed from `cipherstash/encrypt-query-language`. **The directory is the subtree root, not the package.** It was imported at a *verbatim prefix* so its repo-root-relative paths (mise tasks, `Doxyfile`, `sync-generated.mjs`) keep resolving, which puts the npm package `@cipherstash/eql` two levels down at `packages/eql/packages/eql` — the same shape as `packages/protect-ffi/platforms/*`, and enrolled the same way, by an explicit `packages/eql/packages/*` glob in `pnpm-workspace.yaml`. The subtree root deliberately carries no `package.json`. Also contains a **nested Cargo workspace** at `packages/eql/crates/` (`eql-bindings`, published in lockstep with the npm package, plus `eql-domains` / `eql-codegen` / `eql-tests-macros`, which are not), a SQLx test crate at `packages/eql/tests/sqlx`, an ~900-line `mise.toml` task surface, its own `AGENTS.md`, and `docs/`. See the "Working on EQL" notes below before touching it.
87-
**Repository ownership:** EQL now lives in `cipherstash/stack`. File and update
88-
EQL issues in this repository, never in the historical
89-
`cipherstash/encrypt-query-language` repository. Old upstream issue and PR
90-
links are provenance only.
9187
- `e2e/*`: Cross-package end-to-end tests (package managers, supply chain, Prisma example README)
9288
- `examples/*`: Working apps (basic, prisma, supabase-worker)
9389
- `docs/plans/*`: Internal design plans. User-facing documentation lives at https://cipherstash.com/docs (not in this repo).

CLAUDE.md

Lines changed: 0 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -6,22 +6,6 @@ one source of truth.
66

77
@AGENTS.md
88

9-
## Agent skills
10-
11-
### Issue tracker
12-
13-
Issues are tracked in GitHub Issues for `cipherstash/stack`. See
14-
`docs/agents/issue-tracker.md`.
15-
16-
### Triage labels
17-
18-
Use the five default triage labels. See `docs/agents/triage-labels.md`.
19-
20-
### Domain docs
21-
22-
Use a multi-context layout rooted at `CONTEXT-MAP.md`, with package-specific
23-
`CONTEXT.md` files added as needed. See `docs/agents/domain.md`.
24-
259
## Before you finish
2610

2711
Two rules from `AGENTS.md` are the ones most often missed. They are repeated here

CONTEXT-MAP.md

Lines changed: 0 additions & 14 deletions
This file was deleted.

biome.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
{
2-
"$schema": "https://biomejs.dev/schemas/2.5.3/schema.json",
2+
"$schema": "https://biomejs.dev/schemas/2.5.9/schema.json",
33
"files": {
44
"includes": [
55
"**",

docs/agents/domain.md

Lines changed: 0 additions & 50 deletions
This file was deleted.

docs/agents/issue-tracker.md

Lines changed: 0 additions & 60 deletions
This file was deleted.

docs/agents/triage-labels.md

Lines changed: 0 additions & 15 deletions
This file was deleted.

docs/superpowers/specs/2026-08-31-eql-safe-reinstall-design.md

Lines changed: 14 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,8 @@
11
# EQL safe reinstall — durable data and reconstructed indexes
22

3-
Status: proposed
3+
Status: implemented
44
Date: 2026-08-31
55
Issues: cipherstash/stack#959, cipherstash/stack#918
6-
ADR: `docs/adr/0001-eql-data-survives-disposable-schema-reinstall.md`
76

87
## 1. Goal
98

@@ -30,8 +29,8 @@ mechanically stop the operation before the first destructive statement.
3029

3130
### Reconstructable
3231

33-
- Functional indexes whose complete definitions can be obtained with
34-
`pg_get_indexdef()`.
32+
- Standalone functional indexes whose definitions and supported catalog state
33+
can be captured and restored.
3534

3635
### Fail-closed
3736

@@ -61,22 +60,22 @@ lock serializes cooperating EQL lifecycle commands; it cannot serialize
6160
arbitrary DDL issued by unrelated PostgreSQL sessions without superuser-only
6261
event triggers. Application migrations must not run concurrently.
6362

64-
1. Acquire an advisory lock preventing concurrent EQL lifecycle operations.
63+
1. Begin one transaction and acquire the transaction-scoped advisory lock that
64+
serializes EQL lifecycle operations.
6565
2. Discover every customer-owned object with a dependency path to
6666
`eql_v3` or `eql_v3_internal`.
6767
3. Partition dependencies into reconstructable functional indexes and
6868
fail-closed objects.
6969
4. If any fail-closed or unknown dependency exists, print an inventory and exit
7070
before executing installer SQL.
71-
5. Capture each index's identity and `pg_get_indexdef()` output, including
72-
schema-qualified table and index names.
73-
6. Begin one transaction, execute the shipped installer, and recreate captured
74-
indexes before commit. Use the original definition by default; any
75-
concurrent-rebuild mode must account explicitly for PostgreSQL's transaction
76-
restrictions.
71+
5. Capture each index's identity, `pg_get_indexdef()` output, validity/readiness,
72+
clustering, replica-identity role, and comment.
73+
6. Execute the shipped installer, recreate each captured index, and restore its
74+
clustering, replica-identity role, and comment before commit.
7775
7. `ANALYZE` affected tables.
78-
8. Verify every captured index exists, is valid and ready, and still has the
79-
exact server-rendered definition captured before replacement. Query-level
76+
8. Verify every captured index exists; its definition, clustering,
77+
replica-identity role, and comment match; and its validity/readiness have not
78+
regressed. Query-level
8079
engagement remains the responsibility of `stash eql validate`, which has
8180
the application schema needed to construct representative predicates.
8281
9. Commit only after verification, then release the advisory lock.
@@ -97,9 +96,8 @@ The classifier is an allowlist: only ordinary functional indexes with a complete
9796
server-rendered definition are automatically reconstructable. Every unrecognised
9897
class is fail-closed.
9998

100-
Uniqueness, predicates, included columns, tablespaces,
101-
storage parameters, quoting, and non-`public` application schemas require test
102-
coverage before their corresponding index form enters the allowlist.
99+
Each supported index form and catalog property requires regression coverage.
100+
Partitioned index attachment trees remain fail-closed.
103101

104102
## 6. Acceptance criteria
105103

0 commit comments

Comments
 (0)