feat(ci): native CI regression gate over registry verdicts - #7
Merged
Conversation
Add `crucible ci REGISTRY`: capture a sealed baseline of the current verified-latest verdict per (thesis, claim), then gate a later commit by re-deriving the current verdicts and failing nonzero on regression (a claim moving MATCH -> DRIFT, becoming UNVERIFIABLE, or dropping out of the verified-latest set, fail-closed). Emit a deterministic PR-comment-ready Markdown matrix whose every cell references the re-derivable assessment packet seal. The gate rides on the existing witnessed assessments: it re-reads the same verified-latest state `registry stats` uses and opens no second verdict path. The pure core (ci_gate) ranks statuses over two snapshots; the render layer (ci_report) emits the Markdown; the baseline seals its own cells so a hand-edited baseline is rejected on load. Tests assert the three contract behaviors: no-regression exits 0 with a MATCH summary, an injected regression exits nonzero and names the drifted claim, and the Markdown summary is deterministic. README documents the command and ships a ready-to-copy GitHub Action snippet. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Fixes the CI lint gate on this branch; the import fix already landed on main but this branch predated it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this ships
A native CI regression gate:
crucible ci REGISTRY.--write-baseline FILEcaptures the current verified-latest verdict per (thesis, claim) as a sealed baseline snapshot (crucible.ci-baseline/v1).--baseline FILEre-derives the current verdicts, compares them against the baseline, exits nonzero on regression, and emits a deterministic PR-comment-ready Markdown claims-by-rounds verdict matrix.MATCH -> DRIFT, becomingUNVERIFIABLE, or dropping out of the verified-latest set (fail-closed: a claim that lost its witnessed standing is never read as held). New claims and improvements do not fail the gate.crucible verdicts REGISTRY --verify.--out FILEwrites the Markdown while preserving the gate exit code;--jsonemits the full gate report.The feature rides on the existing accountability layer: it re-reads the same
verified-lateststateregistry statsuses and opens no second verdict path. The gate's own output is a pure function of two snapshots, so it recomputes from the record and cannot be asserted.Structure
crucible/ci_gate.py(255 lines) - the pure core:Cell/Snapshot(sealed) /GateReport, status ranking,gate(),cells_from_latest().crucible/ci_report.py(67 lines) - deterministic Markdown rendering (split from logic per the repo'sreport.pyvsassess.pyconvention, keeping every file under the 300-line budget).crucible/ci_cmd.py(85 lines) - thecrucible cicommand handler.cli.py/__init__.py- wiring and public API exports.Docs
if: always()on the comment step so the exit code still gates).Tests (23 new; suite 306 -> 329 passed, 1 skipped)
The three contract behaviors are asserted directly:
test_no_regression_exits_zero_with_match_summary)test_injected_regression_exits_nonzero_and_names_the_drifted_claim, asserting the drifted claim id appears in both stderr and the Markdown)test_markdown_summary_is_deterministic_across_runsand a unit-level determinism test)Plus: dropped-claim regression, drift->unverifiable regression, improvement/new not failing, tampered-baseline rejection, wrong-kind rejection, order-independent seal, JSON output,
--outfile behavior, andcells_from_latestskipping unknown statuses rather than guessing.Test evidence
Shipped vs specced
Fully shipped and tested; nothing deferred to a spec. Zero-dependency and Fair-Source posture preserved (stdlib only). Zero em-dashes / en-dashes. Experimental nothing; all behavior is exercised by tests.
Honest limitations
held.crucible driftremains the margin-delta view.(thesis_id, claim_id); a claim whose id changes (e.g. its content hash shifts) is seen as a drop + a new claim, which is the fail-closed reading.tests/test_browser_evidence.pyhas an unrelated ruff I001 import-order warning that exists onmain; not touched here.Co-Authored-By: Claude Fable 5 noreply@anthropic.com
🤖 Generated with Claude Code