test(v4): naming-completeness guard - section-8 surface sweep, section-9 phase-table agreement, rule-11 consumer coverage - #740
Conversation
…eement + consumer coverage (closes the TODO guard row) tests/test_naming_guard.py enforces the three v4-program naming invariants that were previously prose-only (spec in the module docstring, replacing the TODO.md row): (A) Public-surface sweep - every __all__ export's signatures, dataclass fields (custom-__init__ extras included) and property names checked against the section-8 rename vocabulary; every hit must be a ledger row, structurally exempt (wrapper functions / dying classes), or allowlisted with a stated reason. First run: 0 unrowed violations; 49 documented carve-outs (the CS-cohort groups family, rule-1 canonical calendar time surfaces, rank_control_units' own lambda_reg). A binding self-test forces every param/field rename token - gated rows included - into the sweep predicate. (B) Ledger<->section-9 phase-table agreement in both directions: bracketed citation expansion (single tokens, compound brackets with ranges/comma elements/trailing prose, endpoint-bracket ranges; bare prose ids are NOT citations), the ship-version predicate with the decision_due channel for env-default rows, M-008 positive / M-031 negative fixtures, terminal-row exemption, and a cross-check of the hardcoded map against the table's Ships-in column. (C) Rule-11 consumer coverage over diff_diff/ source, the packaged guides/*.txt, and docs/methodology/, tiered by row kind and token ambiguity: regex lanes (quoted/kwarg/attr/backtick + qualified docs forms) for distinctive tokens; AST call-site matching (callee-aware, including super().__init__ delegation and exported subclasses sharing the effective constructor) plus getattr/hasattr/.get literal state reads for ambiguous tokens like time/group/robust; bare-call text+AST lanes for function rows (docstring examples included, wrapper bodies exempt). Coverage = the token family's code_refs UNION (rule 11's new token-family clause) or a reasoned allowlist entry; both allowlists carry a reachability test that kills dead entries. A version-aware lifecycle gate keyed on __version__ re-arms deferred rows (M-050..M-058, M-016) at each release bump, and terminal rows stay enforced with their own historical code_refs discounted. Accepted limitations (dict/**kwargs indirection, ambiguous attr/dict-key reads) are documented with their measured noise rationale. Parser/matcher/resolver/ gate self-tests run on synthetic fixtures. The guard's first run completed 30 rows' code_refs with newly confirmed readers: diagnostics.py's seven fit(time=...) call sites (M-030), sun_abraham.py's robust=True legacy alias (M-115), synthetic_did.py's super().__init__(robust=True) (M-045), diagnostic_report.py's bare bacon_decompose(...) call and _diagnostic.py's docstring example plus twfe.py's guidance string (M-076), REGISTRY's treatment_col / cohort / controls / robust / clean_control / synthetic_control() / dCDH results.groups references, the packaged-guide readers (cohort= ETWFE examples, aggregation=, wrapper calls, clean_control, placebo_effects, first_treat_col), and record-anyway entries on the lifecycle-gated M-016/M-050 families and the old==new-skipped M-020 family. docs/v4-design.md: rule 11 gains the token-family code_refs-union clause (decided 2026-08-01 over per-row duplication); the section-8 carve-out paragraph adds plot_group_effects' cohort selector; section-9 sequence item 2 and 3.9-cut checklist item 4 now point at the test file. TODO.md's completed guard row is removed. Wired into docs-tests.yml (trigger paths + step) so docs-only ledger/spec edits cannot bypass enforcement.
Overall assessment✅ Looks good — no P0 or P1 findings. Executive summary
MethodologyNo findings. The Methodology Registry content is unchanged; additions only reference existing methodology documents as rename consumers. Code QualityNo findings. PerformanceNo findings. The guard caches file contents and parsed ASTs. MaintainabilityNo findings. Parser and matcher behavior has focused synthetic regression coverage. Tech DebtNo findings. Removing the completed SecurityNo findings. No secrets, unsafe execution, or permission changes were introduced. Documentation/Tests
|
…hases (CI review P3)
|
🔁 AI review rerun (requested by @igerber) Head SHA: Overall assessment✅ Looks good — no unmitigated P0 or P1 findings. Executive summary
MethodologyNo findings. Methodology files are referenced only as naming consumers; their contents and estimator behavior are unchanged. Code QualityNo findings. PerformanceNo findings. MaintainabilityNo findings. The guard includes focused parser, matcher, lifecycle, and terminal-row self-tests. Tech DebtNo findings. Removing the completed TODO entry is consistent with the implemented guard. SecurityNo findings. Documentation/TestsNo findings. The previous accepted-limitation wording is corrected at |
Summary
tests/test_naming_guard.py(~1,150 lines incl. self-tests): mechanical enforcement of the three v4-program naming invariants that were previously prose-only, per the amended TODO.md guard-row spec (now carried in the module docstring). Sequenced as item 2 of v4-design §9's "Remaining 3.9 sequence" - lands BEFORE the 2(c) rename sweep so that PR works from a mechanically-verified list.diff_diff.__all__export's signatures (all public methods, custom dataclass__init__extras), dataclass fields and property names checked against the §8 rename vocabulary; each hit must be a ledger row, structurally exempt (wrapper functions / dying classes), or allowlisted with a stated reason. First run: 0 unrowed violations, 49 documented carve-outs (CS-cohortgroupsfamily, rule-1 canonical calendartimesurfaces,rank_control_units[lambda_reg]). A binding self-test forces every param/field rename token - lifecycle-gated rows included - into the sweep predicate.M-031'sare not citations), the ship-version predicate with thedecision_duechannel for env-default rows (M-008 positive / M-031 negative named fixtures), terminal-row exemption, and a cross-check of the phase→version map against the table's Ships-in column.diff_diff/source, the packagedguides/*.txt, anddocs/methodology/, with lanes tiered by row kind and token ambiguity: regex lanes + qualified docs forms for distinctive tokens; callee-aware AST call-site matching (incl.super().__init__delegation and exported subclasses sharing the effective constructor) plusgetattr/hasattr/.getliteral state reads for ambiguous tokens; bare-call text+AST lanes for function rows (docstring examples included, wrapper bodies exempt). Coverage = the token family'scode_refsunion or a reasoned allowlist entry; a reachability test kills dead entries on both allowlists. A version-aware lifecycle gate re-arms deferred rows at each release bump, and terminal rows stay enforced with their historicalcode_refsdiscounted.docs/v4-deprecations.yaml: 30 rows'code_refscompleted with guard-confirmed readers (diagnostics.pyfit(time=...)sites on M-030,sun_abraham.py'srobust=Truelegacy alias on M-115,synthetic_did.py'ssuper().__init__(robust=True)on M-045,diagnostic_report.py's barebacon_decompose(...)call + docstring/guidance readers on M-076, REGISTRY/guides readers across thecohort/controls/clean_control/aggregation/treatment_col/wrapper families, record-anyway entries on the gated M-016/M-050 families and the old==new-skipped M-020 family).docs/v4-design.md: §8 rule 11 gains the token-familycode_refs-union clause; the §8 carve-out paragraph addsplot_group_effects' cohort selector; §9 sequence item 2 and 3.9-cut checklist item 4 now point at the test file.TODO.md's completed guard row is removed..github/workflows/docs-tests.yml: trigger paths + a dedicated step so docs-only ledger/spec edits cannot bypass the guard (the file also runs in the label-gated main suite). CHANGELOG entry included.Methodology references (required if estimator / math changes)
code_refsmetadata only;docs/methodology/paths are recorded as rename-consumer references, their content is untouched)Validation
tests/test_naming_guard.py(20 tests: 9 guard tests + 11 parser/matcher/resolver/gate self-tests on synthetic fixtures).pytest tests/test_naming_guard.py tests/test_v4_matrix.py: 258 passed, 17 skipped - the ledger edits keep the whole matrix enforcement suite green.Security / privacy