Skip to content

fix: summary-alpha guard sweep, zero-SE plot gate, lambda-slope dedup, shared pscore_trim validator (DML follow-ups) - #795

Merged
igerber merged 8 commits into
mainfrom
dml-followups
Aug 27, 2026
Merged

fix: summary-alpha guard sweep, zero-SE plot gate, lambda-slope dedup, shared pscore_trim validator (DML follow-ups)#795
igerber merged 8 commits into
mainfrom
dml-followups

Conversation

@igerber

@igerber igerber commented Aug 27, 2026

Copy link
Copy Markdown
Owner

Summary

  • Staggered-family summary(alpha=)/print_summary(alpha=) guard (ledger row M-146): eight results classes (CallawaySantAnna, StaggeredTripleDiff, dCDH, Imputation, EfficientDiD, TwoStage, Stacked, SunAbraham) previously did alpha = alpha or self.alpha and relabeled the confidence-interval header at the requested alpha while printing fit-time stored intervals — silent coverage mislabeling (bootstrap percentile intervals cannot be reconstructed from the SE). A non-fit alpha now raises via one shared results_base._require_fit_alpha guard (the DMLDiDResults/EventStudyResults precedent); alpha=0.0, previously swallowed by the falsy or, raises too. Docstrings updated; the non-staggered siblings with the same idiom are tracked as a new TODO.md audit row.
  • plot_event_study zero-SE pointwise gate (the plot_group_effects twin): effect ± z·SE reconstruction NaN-gates zero/negative-SE rows on every route that reaches it, while auto-inferred reference rows (effect 0, se 0) on the raw event_study_effects route retain their degenerate constraint bar per the REGISTRY reference-retention contract. plot_honest_event_study's raw (non-container) routes now mirror the container's retained-row semantics: zero/non-finite-SE rows are excluded up front, the reference period is auto-inferred (attribute or HonestDiD's constraint signature — never a bare -1 fallback), and an all-undefined surface raises instead of rendering a blank figure.
  • Shared utils.validate_pscore_trim (ledger row M-145): promotes the TripleDifference/DMLDiD inline validator and migrates 5 copies. ContinuousDiD tightens from [0, 0.5) to (0, 0.5) (trim=0 disabled the overlap clip keeping 1/(1-p) IPW/DR weights finite) and closes the np.isfinite type hole; CallawaySantAnna gains the type guard at construction and the fit-path re-check; TripleDifference/CS/ContinuousDiD now coerce to built-in float; LWDiD changes message wording only. The deprecated StaggeredTripleDifference deliberately keeps its permissive construction shape (M-013/M-144 posture).
  • Chang (2020) Case 2 λ-slope dedup: DMLDiD's RCS cell loop computes Ĝ₂λ once per cell via an internal _chang_rcs_score_augmented_with_slope variant (previously twice, each with its own validation pass). Public score API and all numerics unchanged (bitwise-pinned by the existing 1e-15/1e-12 oracles plus a new equivalence test).
  • Housekeeping: retires four TODO.md rows (feat(dml): DMLDiD panel=False repeated cross sections - Chang (2020) Case 2 (DML PR-B2) #794 review origins), adds the non-staggered summary-audit row and a DEFERRED.md decision record (no allow_zero flag), updates REGISTRY (M-142 note closure, Event Study Plotting note, CS M-146 note), the v4 deprecations ledger (+2 rows, snapshot/count sync in tests/test_v4_matrix.py and docs/v4-design.md), and docs/migration-4.0.md.

Methodology references (required if estimator / math changes)

  • Method name(s): Chang (2020) Case 2 λ-corrected variance (internal refactor only, score equations unchanged); propensity-score trimming overlap guard (Callaway-Sant'Anna family); event-study reference-period plotting conventions.
  • Paper / source link(s): Chang, N.-C. (2020), "Double/debiased machine learning for difference-in-differences models", Econometrics Journal 23(2) — see docs/methodology/papers/chang-2020-review.md; REGISTRY sections "DMLDiD", "Event Study Plotting", "CallawaySantAnna", TripleDifference M-142 note.
  • Any intentional deviations from the source (and why): None new. The pscore_trim strict-overlap bound and the reference-row retention rule are documented REGISTRY notes (M-145 / Event Study Plotting Note); the summary-alpha guard follows the shipped DMLDiD/EventStudyResults convention (M-146).

Validation

  • Tests added/updated: tests/test_dr_scores.py (internal-variant equivalence), tests/test_dml_did.py (unchanged, re-run), TestSummaryAlphaContract in 8 estimator test files (non-fit alpha, alpha=0.0, fit-alpha acceptance, CS print_summary relay), tests/test_visualization_new.py + tests/test_visualization_plotly.py (zero-SE gate, reference retention across producer conf_int shapes, plotly band-omission), tests/test_event_study_consumers.py (HonestDiD raw-route filter/inference/empty-guard, container explicit-normalization gate), pscore_trim type-guard/coercion/fit-mutation tests in tests/test_continuous_did.py, tests/test_staggered.py, tests/test_v4_merge_ddd.py, tests/test_lwdid.py, and ledger sync in tests/test_v4_matrix.py.
  • Backtest / simulation / notebook evidence (if applicable): N/A — existing bitwise oracles (tests/test_methodology_dr_scores.py atol=1e-15; tests/test_methodology_dml_did.py SE reconstruction at 1e-12/1e-14) pass unchanged, pinning the λ-slope refactor's numerics.

Security / privacy

  • Confirm no secrets/PII in this PR: Yes

…pe dedup, shared validate_pscore_trim (M-145)
@github-actions

Copy link
Copy Markdown

Overall assessment

⚠️ Needs changes — two unmitigated P1 edge-case defects.

Executive summary

  • Chang (2020) Case 2 score and variance equations remain unchanged.
  • Extended-precision propensity trims can silently become zero after validation.
  • HonestDiD can render a reference-only plot when every estimated period has undefined inference.
  • Documented summary-alpha and reference-row conventions are not defects.

Methodology

  1. Severity: P1 — propensity overlap guard can be silently disabled
    Location: diff_diff/utils.py:L521-L528
    Impact: A positive np.longdouble smaller than binary64’s range passes validation, then float(value) underflows to 0.0. All migrated estimators consequently store an invalid zero trim, potentially admitting p=1 into 1/(1-p) weights. This also regresses LWDiD, which previously converted before validating.
    Concrete fix: Convert to float first, then validate the converted value. Add an extended-precision underflow regression test.

  2. Severity: P1 — reference-only HonestDiD surfaces bypass the empty-result guard
    Location: diff_diff/visualization/_event_study.py:L1095-L1111
    Impact: The inferred reference is retained even when every estimated period has zero/non-finite SE, so retained is nonempty and an anchor-only figure is rendered. This contradicts the documented all-undefined rejection in docs/methodology/REGISTRY.md:L5908-L5913.
    Concrete fix: Require at least one defined, non-reference period before plotting an implicitly selected raw surface.

  3. Severity: P3 — documented implementation choices
    Impact: The summary-alpha guard and reference-row retention are explicitly documented. The deduplicated λ-slope still implements both Chang correction terms from docs/methodology/papers/chang-2020-review.md:L126-L134.
    Concrete fix: None.

Code Quality

No additional findings.

Performance

  • Severity: P3 — Computing the λ-slope once per cell is a valid, numerically equivalent improvement. No action required.

Maintainability

No additional findings.

Tech Debt

  • Severity: P3 — Remaining non-staggered summary audits are tracked in TODO.md:L24; the allow_zero decision is tracked in DEFERRED.md:L142. No action required here.

Security

No findings.

Documentation/Tests

  • Severity: P2
    Impact: tests/test_event_study_consumers.py:L2236-L2248 tests all-undefined data only without a reference row, missing the P1 loophole. Propensity tests cover ordinary NumPy floats but not conversion underflow.
    Concrete fix: Add both edge-case regressions described above.

Path to Approval

  1. Validate pscore_trim after conversion and test an underflowing np.longdouble.
  2. Reject raw HonestDiD surfaces containing only a reference anchor, with a test using one reference row plus only zero/NaN-SE estimated rows.

…derflow); reject reference-only honest surfaces (PR #795 review P1s)
@github-actions

Copy link
Copy Markdown

🔁 AI review rerun (requested by @igerber)

Head SHA: d70113c183b09f6e84370a067642c5a257c05bef


Overall assessment

⚠️ Needs changes — one newly identified P1 overlap-guard defect remains.

Executive summary

  • Both previous P1 findings are resolved: conversion underflow is rejected and reference-only implicit HonestDiD surfaces raise.
  • Chang (2020) Case 2 score and variance equations remain unchanged, including both correction terms.
  • [Newly identified] very small positive trims can still make the binary64 upper clip equal exactly 1.0.
  • Summary-alpha and event-study plotting behavior is documented in the Methodology Registry.
  • No security or material performance concerns found.

Methodology

  1. Severity: P1 — [Newly identified] accepted trims can still disable the upper overlap guard
    Location: diff_diff/utils.py:L526-L536; consumers include diff_diff/staggered.py:L3310-L3317 and diff_diff/continuous_did.py:L1416-L1418
    Impact: validate_pscore_trim(1e-20) returns a positive float, but binary64 evaluates 1.0 - 1e-20 as exactly 1.0. Consequently, np.clip(pscore, trim, 1-trim) can retain pscore=1, violating the Registry’s strict-overlap contract and exposing 1/(1-p) computations to division by zero.
    Concrete fix: After conversion, require 1.0 - coerced < 1.0; reject trims that cannot produce a representably strict upper bound.

  2. Severity: P3 — documented methodology choices
    Location: docs/methodology/REGISTRY.md:L2877-L2892, L3624-L3635, L5892-L5913
    Impact: The clipping policy, summary-alpha rejection, and reference-row plotting conventions are documented implementation choices. The deduplicated Chang λ-slope retains both Theorem 2 corrections.
    Concrete fix: None.

Code Quality

No additional findings.

Performance

  • Severity: P3
    Impact: Computing Ĝ₂λ once per cell removes a redundant validation and O(n) pass without changing the equations or numerical result.
    Concrete fix: None.

Maintainability

No findings.

Tech Debt

  • Severity: P3
    Impact: The remaining summary audit and unused allow_zero decision are tracked in TODO.md:L24 and DEFERRED.md:L142; they are not blockers.
    Concrete fix: None required in this PR.

Security

No findings.

Documentation/Tests

  • Severity: P2 — trim regression tests do not verify the derived upper bound
    Location: tests/test_continuous_did.py:L1697-L1713
    Impact: The test asserts only that the returned trim is positive, so values whose 1-trim rounds to 1.0 remain uncovered.
    Concrete fix: Add a regression using 1e-20 and assert rejection; for accepted values, assert 1.0 - result < 1.0.

Targeted pytest execution was unavailable because the review environment lacks pytest and NumPy; source syntax compilation succeeded.

Path to Approval

  1. Reject trims for which 1.0 - coerced == 1.0.
  2. Add the binary64 cancellation regression described above.

@github-actions

Copy link
Copy Markdown

🔁 AI review rerun (requested by @igerber)

Head SHA: f691af9b069900ce7b8c77928a75bbe5b55dea05


Overall assessment

⚠️ Needs changes — one newly identified P1 empty-selection defect remains.

Executive summary

  • Both previous P1 findings are resolved: sub-ULP trims are rejected, and implicit reference-only HonestDiD plots raise.
  • [Newly identified] explicitly selecting only the reference period still bypasses the empty-estimate guard.
  • Chang (2020) Case 2 score and variance equations remain unchanged, including both corrections.
  • Summary-alpha, trimming, and plotting policies are Registry-documented.
  • No security or material performance concerns found.

Methodology

  1. Severity: P3 — documented methodology choices
    Impact: The Chang refactor preserves −Dθ/p̂ + Ĝ₂λ(T−λ̂) and the same variance calculation. Trimming, summary-alpha rejection, and reference-row plotting conventions are documented Registry choices.
    Location: diff_diff/_dr_scores.py:L371-L463; diff_diff/dml_did.py:L1406-L1415; docs/methodology/REGISTRY.md:L2877-L2892, L5892-L5913
    Concrete fix: None.

Code Quality

  1. Severity: P1 — [Newly identified] explicit reference-only selections bypass the empty-result guard
    Impact: _honest_raw_route_periods() verifies that at least one estimated period has valid inference only when periods is None. With an all-undefined surface, periods=[reference_period] passes because the roster is nonempty, then renders a meaningless anchor-only figure—contrary to the Registry’s all-undefined rejection contract.
    Location: diff_diff/visualization/_event_study.py:L1095-L1122
    Concrete fix: Apply if not any(_defined(p) for p in retained): raise ValueError(...) after both branches. Add a regression using a NaN-SE reference, only zero/NaN-SE estimated rows, and periods=[reference_period].

Performance

  1. Severity: P3
    Impact: Computing Ĝ₂λ once per cell removes redundant validation and an O(n) pass without changing numerics.
    Concrete fix: None.

Maintainability

No findings.

Tech Debt

  1. Severity: P3
    Impact: The remaining summary audit and unused allow_zero decision are tracked in TODO.md and DEFERRED.md.
    Concrete fix: None required here.

Security

No findings; no secret-like material was identified.

Documentation/Tests

  1. Severity: P2 — the documented trim domain omits its representability constraint
    Impact: Documentation says every 0 < trim < 0.5 value is accepted, but the corrected validator also requires 1.0 - trim < 1.0 in binary64.
    Location: diff_diff/utils.py:L506-L545; docs/v4-deprecations.yaml:L1749-L1761
    Concrete fix: Document the representably strict upper-bound requirement in M-145 and affected parameter documentation.

Targeted pytest execution was unavailable because NumPy and pytest are absent. Static syntax compilation passed for the modified Python paths.

Path to Approval

  1. Make the HonestDiD valid-estimate check apply to explicit periods= selections and add the reference-only regression described above.

…lections; document the pscore_trim representability constraint (PR #795 review)
@github-actions

Copy link
Copy Markdown

🔁 AI review rerun (requested by @igerber)

Head SHA: ead13e4b122b4a5a4fe8e46de08baba2646dea72


Overall assessment

Looks good — no unmitigated P0 or P1 findings.

Executive summary

  • Previous HonestDiD reference-only P1 is resolved with a regression test.
  • Chang (2020) Case 2 score and variance equations remain unchanged.
  • Summary-alpha guards correctly reject non-fit values, including 0.0.
  • Propensity trimming changes are documented and numerically fail-safe.
  • Runtime tests were unavailable because NumPy/pytest are not installed; modified Python files parse successfully.

Methodology

Severity: P3 — documented methodology choices

Impact: Chang’s (G_{2\lambda}), treated-share correction, and variance calculation are preserved. Trimming and plotting behavior are documented choices.

Location: diff_diff/_dr_scores.py:L346-L464; diff_diff/dml_did.py:L1407-L1445; docs/methodology/REGISTRY.md:L2878-L2892

Concrete fix: None.

Code Quality

No findings. The prior explicit periods=[reference] defect is fixed at diff_diff/visualization/_event_study.py:L1108-L1120.

Performance

Severity: P3 — beneficial refactor

Impact: The Chang lambda slope is computed once per cell instead of twice, without changing results.

Concrete fix: None.

Maintainability

No findings. Shared alpha and trimming validators reduce duplicated policy logic.

Tech Debt

Severity: P3 — tracked

Impact: Remaining non-staggered summary auditing is explicitly tracked in TODO.md; the unused allow_zero decision is recorded in DEFERRED.md.

Concrete fix: None required in this PR.

Security

No findings; no secret-like material was identified.

Documentation/Tests

Severity: P2 — public parameter documentation remains slightly incomplete

Impact: The validator also requires 1.0 - float(trim) < 1.0, while several estimator docstrings describe only (0, 0.5). The detailed constraint is documented in M-145 and the validator itself, so behavior is discoverable but unevenly surfaced.

Location: diff_diff/utils.py:L506-L548; diff_diff/continuous_did.py:L221-L226; docs/methodology/REGISTRY.md:L3624-L3635

Concrete fix: Mention the float64 representability condition in affected pscore_trim parameter documentation.

Runtime pytest execution was unavailable; AST parsing passed for all 33 modified Python files.

@github-actions

Copy link
Copy Markdown

🔁 AI review rerun (requested by @igerber)

Head SHA: 944f472b32634769b671aa9be8f3ddd987af463f


Overall assessment

Looks good — no unmitigated P0 or P1 findings.

Executive summary

  • Previous HonestDiD reference-only P1 is resolved for implicit and explicit selections.
  • Chang (2020) Case 2 score and λ-corrected variance remain mathematically unchanged.
  • Shared propensity trimming validation is consistently propagated and documented.
  • Summary-alpha guards prevent confidence-level mislabeling, including alpha=0.0.
  • All 33 modified Python files parse; runtime tests were unavailable because NumPy/pytest are absent.

Methodology

Severity: P3 — documented methodology choices

Impact: The (G_{2\lambda}) slope, treated-share correction, and SE = sqrt(mean(psi_bar²)/n) remain consistent with Chang’s Theorem 2. Trimming and plotting conventions are explicitly documented Registry notes.

Location: diff_diff/_dr_scores.py:L346-L464; diff_diff/dml_did.py:L1409-L1447; docs/methodology/REGISTRY.md:L2966-L2991; docs/methodology/REGISTRY.md:L5892-L5917

Concrete fix: None.

Code Quality

No findings. The shared alpha and propensity validators remove duplicated policy logic.

Performance

Severity: P3 — beneficial refactor

Impact: DMLDiD now computes and validates the λ slope once per RCS cell without changing numerical output.

Location: diff_diff/_dr_scores.py:L432-L464; tests/test_dr_scores.py:L324-L343

Concrete fix: None.

Maintainability

No findings. Parameter storage, fit-path validation, and summary relays remain consistent.

Tech Debt

Severity: P3 — tracked

Impact: Remaining non-staggered summary auditing is recorded in TODO.md; the unused allow_zero validator option is recorded in DEFERRED.md.

Location: TODO.md:L24; DEFERRED.md:L142

Concrete fix: None required in this PR.

Security

No findings. No secret-like material was introduced.

Documentation/Tests

Severity: P3 — prior documentation finding resolved

Impact: All affected estimator docstrings now state both (0, 0.5) and the float64 representability constraint. Regression tests cover reference-only HonestDiD surfaces, zero-SE plotting, trimming validation, and Chang-score equivalence.

Location: diff_diff/continuous_did.py:L224-L228; diff_diff/dml_did.py:L279-L284; diff_diff/lwdid.py:L462-L466; diff_diff/staggered.py:L450-L455; diff_diff/triple_diff.py:L445-L449; tests/test_event_study_consumers.py:L2236-L2287

Concrete fix: None.

@igerber igerber added the ready-for-ci Triggers CI test workflows label Aug 27, 2026
@github-actions

Copy link
Copy Markdown

🔁 AI review rerun (requested by @igerber)

Head SHA: 8117d82673cb3ee1906fefdb04db633401bb681c


Overall assessment

Blocker — one unmitigated P0 finding.

Executive summary

  • Previous HonestDiD reference-only handling is resolved.
  • [Newly identified] Explicit reference_period can still promote a zero-SE estimated row into a reference anchor and draw a misleading zero-width CI.
  • Chang (2020) Case 2 score and λ-corrected variance remain unchanged.
  • Trimming and summary-alpha behavior are documented Registry choices.
  • All 33 modified Python files parse; runtime tests were unavailable because NumPy/pytest are absent.

Methodology

P0 — Explicit reference bypasses the zero-SE inference gate

Severity: P0

Impact: _honest_raw_route_periods() exempts every row equal to reference_period from its undefined-inference check, without verifying that it is an actual normalization constraint. For example, passing reference_period=0 for an estimated row with effect=1.2, se=0 retains it and reconstructs [1.2, 1.2], despite its stored t-statistic, p-value, and CI being NaN. It also suppresses the honest interval by treating the row as an anchor. This contradicts the Registry’s zero-SE exclusion contract and silently misrepresents inference.

Location: diff_diff/visualization/_event_study.py:L1091-L1118, diff_diff/visualization/_event_study.py:L1303-L1341; docs/methodology/REGISTRY.md:L5908-L5913

Concrete fix: Distinguish a verified normalization anchor from a caller-supplied label. Exempt undefined inference only when the row is confirmed through the result’s reference metadata or constraint signature; otherwise reject or exclude it. Never reconstruct a finite original CI for an undefined-SE row.

P3 — Documented methodology choices

Severity: P3 informational

Impact: Chang’s (G_{2\lambda}) term, treated-share correction, and variance remain consistent with Theorem 2. Propensity clipping and reference-row retention are documented implementation choices.

Location: diff_diff/_dr_scores.py:L346-L464; diff_diff/dml_did.py:L1409-L1447; docs/methodology/papers/chang-2020-review.md:L114-L139

Concrete fix: None.

Code Quality

No additional findings.

Performance

Severity: P3 informational

Impact: The DML RCS path now computes the λ slope once per cell without changing results.

Concrete fix: None.

Maintainability

No findings.

Tech Debt

Severity: P3 informational

Impact: Remaining non-staggered summary auditing and the unused validator option are tracked.

Location: TODO.md:L24; DEFERRED.md:L142

Concrete fix: None required here.

Security

No findings or secret-like additions detected.

Documentation/Tests

The new tests cover inferred references and reference-only selections, but not an explicit reference_period pointing to an estimated zero-SE row.

Path to Approval

  1. Require a verified constraint anchor before exempting reference_period from the zero/non-finite-SE gate.
  2. Add raw MPD and event_study_effects tests where explicit reference_period targets a zero-SE non-anchor, covering default and explicit periods selections.

@igerber igerber removed the ready-for-ci Triggers CI test workflows label Aug 27, 2026
…ference exemption (PR #795 review P0)

An explicit reference_period= label pointing at an ESTIMATED zero-SE row
promoted it past _honest_raw_route_periods' undefined-inference gate,
drawing a zero-width original CI at a nonzero effect and suppressing its
honest interval. The exemption now requires effect exactly 0.0 (every
producer's true normalization row), mirroring plot_event_study's
reference carve-out conjunct. Regression tests cover both raw routes,
default and explicit periods selections, and the legitimate
explicit-anchor case; REGISTRY Note updated.
@github-actions

Copy link
Copy Markdown

🔁 AI review rerun (requested by @igerber)

Head SHA: 69cfa6a047035251bdfd000c82ca98ab530f92c3


Overall assessment

Blocker — the previous P0 is only partially resolved.

Executive summary

  • Nonzero zero-SE estimated rows are now excluded.
  • Zero-effect zero-SE estimated rows can still be mistaken for reference anchors.
  • Chang (2020) score and variance equations remain unchanged.
  • Trimming and summary-alpha changes are documented Registry policies.
  • All 33 changed Python files parse; tests could not run because NumPy is unavailable.

Methodology

P0 — Effect equality does not verify a normalization anchor

Severity: P0

Impact: _is_anchor() treats any caller-selected row with effect == 0.0 as a reference constraint. An estimated row can legitimately have effect=0, se=0 while retaining positive n_groups/n_obs. Such a row bypasses the undefined-inference gate, receives a finite [0, 0] original CI, and has its HonestDiD interval suppressed. This silently converts undefined inference into a normalization constraint and contradicts the Registry’s claim that estimated zero-SE rows cannot be promoted.

Location: diff_diff/visualization/_event_study.py:L1094-L1118, diff_diff/visualization/_event_study.py:L1262-L1291, diff_diff/visualization/_event_study.py:L1315-L1353; docs/methodology/REGISTRY.md:L5908-L5916

Concrete fix: Pass verified anchor provenance into _honest_raw_route_periods. For dictionary surfaces, require the full constraint signature such as zero count, zero effect, and non-finite SE. For MPD surfaces, never treat a caller-supplied label as an anchor without result metadata. Use that verified status—not effect == 0—when retaining rows and suppressing HonestDiD bounds.

P3 — Documented methodology choices

Severity: P3 informational

Impact: The Chang Case 2 implementation still includes both finite-dimensional corrections and matches Theorem 2. Strict propensity clipping and fit-alpha-only summaries are documented Registry policies.

Concrete fix: None.

Code Quality

No additional findings.

Performance

P3 — Lambda-slope deduplication

Severity: P3 informational

Impact: The RCS cell path now computes (G_{2\lambda}) once without changing the score or variance.

Concrete fix: None.

Maintainability

No additional findings.

Tech Debt

P3 — Deferred work is tracked

Severity: P3 informational

Impact: Remaining summary auditing and the validator API decision are recorded in TODO.md:L24 and DEFERRED.md:L142.

Concrete fix: None.

Security

No findings.

Documentation/Tests

P0 — Blocking case remains untested

Severity: P0, same blocker as above

Impact: Tests cover a nonzero estimated row and a genuine zero-effect constraint, but not an estimated effect=0, se=0 row with positive observation/group count.

Location: tests/test_event_study_consumers.py:L2284-L2351

Concrete fix: Add CS-dictionary and MPD tests where explicit reference_period targets an estimated effect=0, se=0 row, covering both implicit and explicit period selection.

Path to Approval

  1. Require verified constraint provenance before applying the reference exemption.
  2. Add the zero-effect estimated-row tests described above.

… on honest plot routes (PR #795 review P0)

effect == 0.0 alone still let an ESTIMATED effect-0/se-0 row with a
positive group count be promoted to a reference anchor by an explicit
reference_period= label. _honest_raw_route_periods now takes the
VERIFIED anchor resolved per route: dict surfaces require the producers'
full constraint signature (zero n_groups/n_obs count, effect exactly
0.0, no defined SE - NaN or 0.0 markers), MPD surfaces verify only
against the result's own reference_period metadata, and the container
route only against is_reference-marked rows. Honest-bound suppression is
keyed on the same verified anchor, so a mislabeled estimated row keeps
its computed honest interval. Regression tests cover both raw routes,
implicit and explicit selection; REGISTRY Note updated.
@github-actions

Copy link
Copy Markdown

🔁 AI review rerun (requested by @igerber)

Head SHA: da2258892414132be16e30ad1526966d3ac61f64


Overall assessment

Looks good — no unmitigated P0 or P1 findings.

Executive summary

  • The previous P0 is resolved: only verified normalization anchors bypass the zero-SE gate.
  • Chang (2020) Case 2 score and variance equations remain unchanged.
  • Trimming and summary-alpha changes are documented Registry policies.
  • Focused tests could not run because NumPy, SciPy, and pytest are unavailable; all 33 changed Python files parse successfully.

Methodology

P3 — Documented methodology choices

Severity: P3 informational
Impact: Propensity clipping, fit-alpha-only summaries, and reference-row plotting behavior are documented Notes in REGISTRY.md. The Chang λ-slope refactor preserves both finite-dimensional corrections.
Concrete fix: None.

P3 — Previous P0 resolved

Severity: P3 informational
Impact: Dictionary anchors now require zero count, zero effect, and undefined SE; MPD and container routes require producer metadata. Honest bounds use the same verified anchor status.
Location: diff_diff/visualization/_event_study.py:L1075-L1126, L1242-L1314, L1346-L1374
Concrete fix: None.

Code Quality

No findings.

Performance

P3 — λ-slope deduplication

Severity: P3 informational
Impact: The RCS cell path computes (G_{2\lambda}) once without altering numerical operations.
Location: diff_diff/_dr_scores.py:L371-L464, diff_diff/dml_did.py:L1408-L1416
Concrete fix: None.

Maintainability

No findings.

Tech Debt

P3 — Remaining work is tracked

Severity: P3 informational
Impact: The non-staggered summary audit and validator API decision are recorded in TODO.md:L24 and DEFERRED.md:L142.
Concrete fix: None.

Security

No findings.

Documentation/Tests

P3 — Regression coverage added

Severity: P3 informational
Impact: Tests now cover estimated effect=0, se=0 rows with positive counts on dictionary and MPD routes, for implicit and explicit selection. Runtime execution was unavailable in this environment.
Location: tests/test_event_study_consumers.py:L2334-L2385
Concrete fix: None; confirm the focused suite passes in CI.

@igerber igerber added the ready-for-ci Triggers CI test workflows label Aug 27, 2026
@igerber
igerber merged commit df187a7 into main Aug 27, 2026
39 of 40 checks passed
@igerber
igerber deleted the dml-followups branch August 27, 2026 20:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-ci Triggers CI test workflows

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant