feat: add LWDiD estimator (Lee & Wooldridge 2025, 2026) - #588
feat: add LWDiD estimator (Lee & Wooldridge 2025, 2026)#588gorgeousfish wants to merge 8 commits into
Conversation
9712477 to
8c5ccce
Compare
|
@gorgeousfish Thanks for this - it's a serious, well-prepared contribution, and the method is a great That said, before I do a full review, one thing to clear first: licensing. This implementation
|
|
Thanks for the quick response and the positive signal. On licensing: Yes, I hold the copyright to all code in this PR. It's my own independent implementation, not derived from Lee & Wooldridge's Stata lwdid package or any third-party source. I'm contributing it under diff-diff's MIT license. On datasets: smoking.csv: Abadie, Diamond & Hainmueller (2010), California tobacco control program. Publicly available, widely redistributed in academic packages. castle.csv: Cheng & Hoekstra (2013) / Cunningham (2021), Castle Doctrine laws. Publicly available. walmart.csv: county-level panel from Brown & Butts (2025, Journal of Econometrics), constructed from County Business Patterns (CBP) data. Publicly available government statistical data. |
|
@gorgeousfish Thanks for the licensing confirmation - that closes the question. Since then we've completed a full evaluation on our side: fresh reviews of both papers against their current SSRN revisions, an independent replication of your headline results, and a code review against the papers and this library's conventions. The short version: this is a strong contribution and we want to merge it. We reproduced your Prop 99 numbers ourselves (demeaning ATT -0.4222, SE 0.1208; detrending -0.2270, SE 0.0941, with the exact-inference p-value matching the paper), and the core is exactly right: the transformation reproduces our Getting it merged takes real work on both sides, so below is the full plan - split, sequenced, and complete (no surprise rounds later). None of it starts until you confirm you're on board. If any part looks wrong to you, push back and we'll discuss. First, one decision from you: the event studyThe current revision's Appendix D event study (placebo/dynamic WATT(r)) plus the Algorithm 1 influence-function multiplier bootstrap (simultaneous sup-t bands) is the one substantial piece of the papers not yet implemented - and it's also this estimator's proper pre-trends diagnostic. Your call:
Both options are genuinely fine with us. The planStep 0 - you, now. Reply confirming (a) you're good with this plan and (b) your event-study choice above. Nothing below starts until then. Step 1 - us. We open a maintainer PR to main with (a) our methodology notes for both papers, written against the current SSRN revisions (June 8 and February 3, 2026 - both papers were revised after you wrote your implementation notes, and the June revision reworked the event-study and inference appendices), and (b) Step 2 - you. Work the fix list below, flipping xfails as you go. One consequence of step 1 to fold into your work: our methodology notes supersede the two review docs currently in this PR - please drop those during the rebase and treat the maintainer versions as canonical (the file paths collide anyway). Step 3 - us. Wild-bootstrap dedupe into the existing The step 2 fix list
Thanks again - the core here is exactly right, which is the hard part. The rest is the normal cost of moving a standalone package into a library with strong invariants, and we're glad to carry our share of it. |
|
Thanks for the thorough plan - I'm on board. On the event study: Option A - I'd like to include the Appendix D event study (WATT(r) + Algorithm 1 multiplier bootstrap) in this PR so LWDiD ships as a standard estimator. Ready for Step 1 whenever you are. I'll start looking into the IPW centering issue in the meantime. |
…on rubric) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FZK3FD9jxWGxBrPDw5APSg
…D REGISTRY entry + references (precursor to #588) Loaders download the MIT-licensed ancillary datasets of the authors' Stata lwdid package (SSC): pinned SHA-256 verification of every byte-load (HTTP-only host), stale-cache re-download, structural validation against source invariants, loud UserWarning + df.attrs['source'] marker on synthetic fallback, seeded local-RNG fallback constructors. REGISTRY.md gains the maintainer-authored LWDiD section (E.1 contributing-unit WATT weights; provenance pinned via reviewed-PDF SHA-256 + live-verified SSRN metadata). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FZK3FD9jxWGxBrPDw5APSg
…il PR #588 lands) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01FZK3FD9jxWGxBrPDw5APSg
8c5ccce to
886b106
Compare
Maintainer rebase onto current main (igerber, 2026-07-17), per plan agreed in PR igerber#588: dropped committed datasets (tutorial now uses the checksummed load_prop99()/load_walmart() loaders on main), kept the maintainer-authored paper reviews and references entries from igerber#685, removed the lwdid dev dependency (external reference implementations stay environmental, importorskip-gated), renumbered tutorial 26 -> 27. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
@gorgeousfish - as promised, here is the detailed file:line review. Three pieces of news first:
What replicates today, verified first-hand on your rebased head: all three Prop 99 donor-pool tables (Table 3: demean -0.4222/0.1208, detrend -0.2270/0.0941; Table 4 Southern; Table A1 Midwestern), per-period effects, and the exact-t p-value (0.0209 vs the paper's printed 0.021). The transformation core is genuinely solid. The items below are ordered by how much they matter. 1. IPW influence function is un-centered (confirmed bug)
The control term is linear in raw Acceptance: 2. Staggered path: per-(g, r) cells + the composite-outcome regression (7.18)/(7.19)This is the deepest item, with two coupled halves. (a) Cell construction. (b) Aggregation and SE. Please also add a hand-built staggered test in your test files asserting period-specific control eligibility directly (e.g. a panel where one later cohort is a valid control at r = 3 but not at r = 5, with the cell samples asserted explicitly) - our golden-based suite catches the SE divergence but not the eligibility semantics on its own. Two sub-points discovered while calibrating:
Acceptance: 3. Event study (your Option A): Appendix D + Algorithm 1The API spec we agreed to is written as normative docstrings in Full per-period goldens for both Walmart outcomes are committed at Related guard: Acceptance: 4 4. Randomization inference convention (discussion item, not necessarily a bug)
5. Silent-failure sites (library policy: never drop/alter user data without a warning)
5b. Design validation gaps
Related: 6. Exceptions: fold into house style
7. API surface: trim top-level exportsKeep 8. Wild cluster bootstrap: we will fold this into house machinery (our take-on)FYI only: 9. Diagnostics triage (as agreed)
10. Typing and tests
Sequencing from here is as agreed in the plan comment: you work down this list on the rebased branch (small commits are easier for us to sync into the mirror for CI review rounds); the xfail suite tracks progress objectively; once it is green-with-no-xfails we do the final REGISTRY/experimental-surface pass and the merge mechanics on our side. The two of the seven step-2 items already taken off your plate: packaging (done in the rebase) and the wild-bootstrap dedupe (our step 3). |
|
I have opened a focused dependent PR for #733: gorgeousfish#1. It targets this branch and preserves the overall ATT and inference already computed by It also adds a slow strict-xfail bootstrap concordance test for the separate covariance-aware inference follow-up, #735. The default not-yet-treated identification defect in #734 remains the merge-blocking item; I have not attempted to fold that larger |
Maintainer rebase onto current main (igerber, 2026-07-17), per plan agreed in PR igerber#588: dropped committed datasets (tutorial now uses the checksummed load_prop99()/load_walmart() loaders on main), kept the maintainer-authored paper reviews and references entries from igerber#685, removed the lwdid dev dependency (external reference implementations stay environmental, importorskip-gated), renumbered tutorial 26 -> 27. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Fix IPW SE centering (translation-invariant influence function) - Implement composite regression (Eq 7.18/7.19) for staggered ATT - Add N_infinity >= 2 guard for never-treated controls - Implement event study (Appendix D): WATT(r) + Algorithm 1 sup-t bands - Fix randomization inference p-value convention (strict > comparison) - Add design validation (absorbing treatment, time-invariant cohort) - Migrate custom exceptions to ValueError - Trim top-level exports to LWDiD/LWDiDResults/LW Acceptance: 39 pass / 9 xfail (IPWRA event-study variants pending). Remaining xfails are IPWRA-specific event-study cases for Step 3.
- Execute all 30 code cells with full outputs (text + plots) - Add missing top-level exports to __init__.py: randomization_inference, wild_cluster_bootstrap, test_parallel_trends, sensitivity_analysis, recommend_transformation
…puts - Fix IPWRA event-study golden tests (pass controls in test helper) - Resolve mypy type errors to zero (type: ignore for pandas Union types) - Execute tutorial notebook with outputs (30/30 cells) - Fix tutorial imports to use module-level paths (per export trim) - Remove stale XFAIL_IPW_CENTERING marker Methodology tests: 43 pass / 5 xfail (non-strict bootstrap SE only). All strict acceptance criteria met.
LWDiDResults inherits BaseResults and AggregationMixin, so aggregate()
follows the library-wide contract (_AGGREGATE_SUPPORTED plus
_aggregate_compute) instead of a private signature.
aggregate("simple") now reports the estimand fit() already computed. The
previous aggregate(by="overall") rebuilt the overall ATT as a
treated-mass-weighted average of the marginal cohort effects and took its
SE as sqrt(sum w_g^2 se_g^2) - an independence assumption across cohorts
that share control units. For the same estimand that returned a narrower
SE than fit() and dropped df_inference entirely, so the aggregated object
reported a different significance level than the fit it came from
(igerber#733).
aggregate("event_study") returns the shared EventStudyResults container
rather than a private dict (igerber#732). The anchor period is
carried as an explicit is_reference row at att = 0 instead of being
dropped, and the sup-t band and its critical value thread through.
New fields for the staggered path to populate: inference_basis (which
basis produced the overall SE, surfaced in summary()),
cohort_time_effects, event_study_vcov/_index/_df, and reference_periods.
to_dict() serialises all of them, so a round-tripped result no longer
loses the event study or the basis of its inference.
Co-authored-by: Cursor <cursoragent@cursor.com>
The staggered path applied control eligibility as a unit-level filter and then averaged whatever calendar window each cohort happened to keep. On a panel whose only signal is a common time trend that produced ATT ~ 1.0 with SE ~ 5e-16: the cohort means were taken over different calendar periods, so the trend itself entered the contrast, and the SE collapsed because the residual variation had been differenced away (igerber#734). Estimation moves to diff_diff/lwdid_staggered.py and runs one cell per (g, t), each on its own cross-section with period-specific control eligibility: never-treated units plus {G > max(g, t)}, i.e. the LW 2026 Appendix D.3 rule evaluated at t rather than once per cohort. A pure common time trend now returns ATT = 0, and adding a constant shift to every unit-period leaves the ATT unchanged. Cells that cannot be identified (no treated or no control units after the eligibility filter, too few pre-periods for the transformation) are recorded with their reason instead of being silently averaged over, and a cohort with no supported cell is dropped rather than contributing a NaN. Overall inference no longer assumes cohorts are independent (igerber#735). Every estimator returns a per-observation influence function through _dispatch_estimator; cell influence functions are accumulated to units, combined with the treated-mass weights, and the overall SE is read off the joint function, so the covariance induced by shared controls is carried rather than dropped. The basis is reported explicitly in inference_basis rather than left implicit: composite_regression never-treated + RA + classical + no covariates, where LW 2026 eq. 7.18/7.19 applies and _composite_regression_aggregation (kept for exactly this) gives the paper's numbers joint_influence_function everywhere else unavailable_matching psm has no influence-function representation; SE is NaN and warns, pointing at ipwra unavailable_degenerate_cells a cell SE is degenerate or non-finite; SE is NaN and warns, naming the cohorts Removes _fit_staggered, _fit_event_study, _aggregate_cohort_effects and the _es_transform_* helpers, all superseded by the cell construction (-929 lines in lwdid.py). The Walmart event-study SE goldens are now all six non-strict rather than an enumerated subset. Changing which controls enter each cell moves the bootstrap SEs slightly in both directions - detrend-ra/a5_wholesale goes 0.0570 -> 0.0581 against a printed 0.057 while demean-ipwra/a4_retail moves into agreement - and at B = 999 the Monte Carlo error on a bootstrap SD is about 2.2%, larger than the 1.9% gap. Which individual columns land inside three-decimal precision is therefore not a stable property to encode. PRINTED_ATOL is unchanged and the point-estimate goldens remain exact checks. Co-authored-by: Cursor <cursoragent@cursor.com>
…te contract Four groups, one per issue in this round. TestCohortTimeCellSupport pins the (g,t) construction (igerber#734): eligibility is period-specific rather than per-cohort and matches never-treated plus {G > max(g,t)} for every cell; a pure common time trend gives ATT = 0; a constant shift applied to every unit-period leaves the ATT unchanged; unsupported cells are reported with a reason and a cohort with no supported cell is dropped. TestInfluenceFunctionReconciliation checks that the influence function each estimator returns actually reproduces that estimator's reported SE, with and without covariates, and that psm reports no influence function instead of a silently independent one. TestStaggeredJointInference covers the overall SE (igerber#735): the joint basis is reported, the SE is wider than the cohort-independence formula it replaces, and it agrees with a unit-cluster bootstrap. TestAggregationContract covers igerber#733 and igerber#732: aggregate("simple") reproduces fit() exactly including df_inference and across every inference basis, "group" carries cohort effects on the shared schema, "event_study" returns EventStudyResults with the anchor as a reference row and survives to_dict(), and the rejected cases (unknown type, weights selector, balance_e off event_study, common-timing fits) all fail closed. test_simple_preserves_the_fitted_result and test_matches_unit_cluster_bootstrap are adapted from Charles Shaw's patch in #1, which independently identified the aggregate() SE discrepancy behind igerber#733. Co-authored-by: Charles Shaw <shawcharles@users.noreply.github.com> Co-authored-by: Cursor <cursoragent@cursor.com>
…ication note REGISTRY.md records why the staggered replication goldens pass control_group='never_treated' explicitly. The implementation default is 'not_yet_treated', matching OVLS (eq. 4.10) as the text states, but the paper's printed staggered results are computed against the never-treated pool only. That is a sample-definition choice the text leaves to the analyst while the applications fix it, not a discrepancy in either direction; a default-pool fit gives different and equally valid estimates because the (g,t) cells draw on a strictly larger control sample. doc-deps.yaml adds diff_diff/lwdid_staggered.py to the lwdid group and gives it the same drift-risk and doc targets as lwdid.py, so the new module is not invisible to the drift check. The 27_lwdid tutorial is re-executed against the (g,t) path so its outputs match what the estimator now returns. Co-authored-by: Cursor <cursoragent@cursor.com>
c2694ff to
10c900a
Compare
|
This round addresses the four issues @shawcharles filed against this branch. Rebased onto Commits
#734 — staggered control eligibility (merge-blocking)The old path applied control eligibility as a unit-level filter and then averaged whatever calendar window each cohort happened to retain, so the cohort means were taken over different calendar periods and a common time trend entered the contrast directly. On a panel whose only signal is that trend it returned ATT ≈ 1.0 with SE ≈ 5e-16. Estimation now runs one cell per #735 — overall staggered standard errorCohort effects that share control units are correlated, so the previous The basis is now reported explicitly in
The two NaN cases fail closed rather than falling back to an independence assumption. #733 —
|
This PR adds native support for the Lee & Wooldridge (2025, 2026) rolling-transformation difference-in-differences method. The approach works by applying unit-specific time-series transformations (demeaning or detrending) to panel outcomes before treatment, converting the panel DiD problem into a standard cross-sectional one. Once transformed, any treatment-effect estimator — regression adjustment, inverse probability weighting, doubly robust, or propensity-score matching — can be applied directly to the cross-section. The method handles both common-timing and staggered-adoption designs with flexible control-group selection.
The implementation lives entirely within
diff_diff/lwdid*.py(9 modules) and introduces zero new runtime dependencies — it reuses the existingsolve_ols,solve_logit, andsafe_inferenceinfrastructure. The IPW and IPWRA standard errors use the full semiparametric influence function with propensity-score and outcome-model correction terms, matching the variance formula in the authors' Stata package.Numerical correctness has been validated against the reference lwdid Python package across all supported configurations. The RA path achieves machine-precision agreement (≤1e-10), and IPW/IPWRA paths agree to within 1%. The California Proposition 99 results from Table 3 of LW (2026) are reproduced exactly: demeaning ATT = −0.422, detrending ATT = −0.227.
Beyond core estimation, the PR includes wild cluster bootstrap inference (Rademacher/Mammen/Webb), Fisher randomization inference, parallel-trends pre-testing, sensitivity analysis, clustering-level diagnostics, and visualization methods. A tutorial notebook walks through the full workflow on the papers' empirical datasets (California smoking data, Castle Doctrine laws, Walmart county-level entry).
Methodology references (required if estimator / math changes)
Validation
docs/tutorials/26_lwdid.ipynb) reproduces Tables 3–4 from LW (2026) on California Proposition 99 data. ATT values match published results to 0.04% precision.Security / privacy