Add WooldridgeDiD opt-out for comparison-support filtering (#820)

Author: igerber

TODO.md

@@ -56,7 +56,6 @@ Related tracking surfaces:
 | `ContinuousDiD` CGBS-2024 remaining extensions (earlier phases — `covariates=` reg/dr, `treatment_type="discrete"`, single-cohort `control_group="lowest_dose"` with estimand `ATT(d)−ATT(d_L)` — are already supported; see REGISTRY Note #7). Remaining (all deferred `NotImplementedError`, documented): `estimation_method="ipw"` on the dose curve (scalar-adjustment / degenerate); `covariates=` × `survey_design=` (weighted OR + weighted nuisance IF); multi-cohort **heterogeneous-support** discrete aggregation (support-aware: average each dose only over the cohorts that observe it); **multi-cohort `lowest_dose`** (within-cohort `d_L` reference + support-aware cross-cohort aggregation); and **`covariates=` × `lowest_dose`** (conditional-PT-relative-to-`d_L` estimand). Single-cohort / 2-period / shared-support multi-cohort are supported. | `continuous_did.py` | CGBS-2024 | Heavy | Low |
 | `WooldridgeDiD` does not apply the W2025 Sec 5.4 `D_{G_max} x X` covariate normalization, and three sibling covariate rank deficiencies are pre-existing. Measured with the period range pinned and only the never-treated units toggled: (1) time-invariant `exovar` is absorbed by the unit FE, 4 of 26 columns, IDENTICALLY with and without never-treated units; (2) `xgvar`'s cell x covariate block, 19 of 41, identical on both panels; (3) `xtvar` under `demean_covariates=False` does exhibit the `sum_g D_g x = x` dependency that the default demeaning removes; (4) the newly-reachable case -- time-VARYING data passed through `exovar`, which its own docstring reserves for time-invariant covariates -- where the paper's `dT_i` rule would give a deterministic `D_{G_max} x X` drop instead of QR's arbitrary pick (coefficients unaffected, `1.35e-14`; `rank_deficient_action="error"` raises). REGISTRY's narrowed Sec 5.4 note cross-references this row. **Trap for whoever takes it:** `xtvar` under the DEFAULT `demean_covariates=True` is FULL RANK -- the raw block carries demeaned values while `D_g x X` carries raw ones -- and forcing the drop there moves `overall_att` 1.11903 -> 1.46269. Pinned as-is by `TestComparisonSupportFiltering::test_cells_derived_groups_did_not_leak_into_the_design`. | `diff_diff/wooldridge.py` | #729-followup | Heavy | Medium |
 | `WooldridgeDiD.n_control_units` counts never-treated UNITS on `control_group="never_treated"` regardless of method, but on the nonlinear paths (`logit`/`poisson`) treated units' pre-treatment rows ARE the identifying comparison -- only the OLS path absorbs them into their own cells. So the reported count under-states the comparison pool exactly where the REGISTRY control-pool asymmetry note applies. Widen to `not_yet_treated or (never_treated and method != "ols")`, or document the count as never-treated-units-by-definition. Behavior is PRE-EXISTING; documented for now in the REGISTRY control-pool Note rather than changed, because widening moves a public results field and wants its own ledger row and test matrix. | `diff_diff/wooldridge.py` | #729-followup | Mid | Low |
-| `WooldridgeDiD` has no opt-out for comparison-support period filtering: a user who would rather see the refusal than a reduced sample cannot ask for it. Adding one means a constructor parameter (`get_params`/`set_params` propagation, transactional validation), a ledger row, and a test matrix across both predicate branches and all three `rank_deficient_action` modes -- deliberately out of scope for the change that introduced the filter. The always-on warning is the interim answer. | `diff_diff/wooldridge.py` | #729-followup | Mid | Low |
 | Bad-control imputation estimator (Caetano et al. 2026 Section 6.1, Eqs. 5-7, S8 influence function) on a CallawaySantAnna `estimation_method="reg"` host: two untreated-sample OLS fits per cell plus the generated-regressor IF line. | `staggered.py` | bad-controls PR-B | Heavy | Low |
 | Bad-control SC "parallel trends for X" variant (Caetano et al. 2026 Section 7 / S17): a linearity-based alternative to covariate unconfoundedness with its own estimand. | `dml_did.py` | bad-controls PR-B | Heavy | Low |
 | `ATT_X(g,t)` event-study aggregation + bootstrap replay for the bad-control pre-test (today analytical per-cell only; never aggregated). | `dml_did_results.py` | bad-controls PR-B | Mid | Low |

changelog.d/20260905-wooldridge-unsupported-period-action.md

@@ -0,0 +1,7 @@
+### Added
+- **WooldridgeDiD comparison-support policy** ([M-147]): set
+  `unsupported_period_action="error"` to refuse periods lacking eligible comparison
+  support before removing them. The default `"drop"` preserves filtering and warnings.
+  The option works across OLS, logit and Poisson independently of
+  `rank_deficient_action`; results record the fit-time policy in `summary()` and
+  `to_dict()`. Existing survey and identification checks remain active.

diff_diff/guides/llms-autonomous.txt

@@ -573,13 +573,14 @@ When `has_never_treated == False`:
 - `ChaisemartinDHaultfoeuille` - constructs switchers vs. non-switchers
   directly; no never-treated requirement.
 - TWFE / `MultiPeriodDiD` / `ImputationDiD` / `TwoStageDiD` /
-  `StackedDiD` / `WooldridgeDiD` - use the last-treated or untreated-
+  `StackedDiD` - use the last-treated or untreated-
   until-late units as implicit controls; estimators do not error, but
   consider whether the implicit control structure is what you want.
 - `WooldridgeDiD` specifically: use `control_group="not_yet_treated"`
   (the default). `control_group="never_treated"` raises when no
-  cohort-0 units exist. On an all-eventually-treated panel the last
-  cohort becomes the reference per W2025 Section 5.4, so periods at
+  cohort-0 units exist. With the default `unsupported_period_action="drop"`,
+  the last cohort becomes the reference on an all-eventually-treated panel
+  per W2025 Section 5.4, so periods at
   which every unit is treated carry no identified ATT(g, t) and are
   REMOVED from the estimation sample before the solve. The fit emits a
   `UserWarning` naming the dropped periods, the observation count and
@@ -588,6 +589,19 @@ When `has_never_treated == False`:
   cohorts. If your agent surfaces warnings to a user, surface these:
   the estimate is computed on fewer rows than were supplied. Stata
   `jwdid` performs the same reduction but reports only a smaller `N`.
+  Use `WooldridgeDiD(unsupported_period_action="error")` when the user prefers
+  refusal to automatic period filtering: it raises `ValueError` naming the
+  unsupported periods and affected observation count before removal. A period
+  lacks support when no positive-weight eligible comparison is observed:
+  never-treated rows only on OLS + `never_treated`, and also rows before
+  `g - anticipation` on other paths. The policy applies to all methods,
+  independently of `rank_deficient_action`; other identification checks and
+  unidentified-cohort exclusion remain active. Only existing pre-filter checks
+  retain precedence; later covariate, nonlinear-outcome, or explicit-cluster
+  validation can be preempted by unsupported-period refusal. With `survey_design`,
+  its pre-filter design validation still runs first: `"error"` then raises `ValueError`, while
+  `"drop"` retains the survey-domain `NotImplementedError` when periods would
+  be removed. Fully supported survey fits are unaffected.
   SOME covariate specifications on such a panel are still rank-deficient
   (`exovar`, `xgvar`, and `xtvar` with `demean_covariates=False`), because
   `D_{G_max} x X` is not normalized; `rank_deficient_action="error"`

diff_diff/guides/llms-full.txt

@@ -1630,13 +1630,29 @@ WooldridgeDiD(
                                             # carries G-1 entries. Rejected with survey_design= and
                                             # with control_group="never_treated".
     df_convention: str = "residual",        # OLS analytical t/p/CI df (3.9: default-hc1 arms moved z -> t(residual)): "residual" (default), "cluster" (G-1, hc1-clustered only), "normal" (z); survey/BM DOF keep precedence; GLM arms knob-independent (explicit non-default warns); flips at v4
+    unsupported_period_action: str = "drop", # "drop" warns and filters; "error" refuses unsupported periods
 )
 ```
 
 `fit()` additionally accepts `survey_design=` (a `SurveyDesign`) on all three
 methods; see the Survey Support section.
 
-**All-eventually-treated panels (no never-treated group).** Use
+**Comparison-support policy.** `unsupported_period_action="drop"` preserves
+period filtering and its warnings. Set `unsupported_period_action="error"`
+to raise `ValueError` naming the unsupported periods and affected observation
+count before removal. Support requires positive-weight never-treated rows on
+OLS + `control_group="never_treated"`; elsewhere not-yet-treated rows before
+`g - anticipation` also qualify. This is independent of `rank_deficient_action`
+and does not control unidentified-cohort exclusion or bypass identification checks.
+Only existing pre-filter configuration, cohort, and survey-design checks retain
+precedence. Later checks (covariate columns, nonlinear outcomes, and non-Conley
+explicit cluster columns) are not preflighted; unsupported-period refusal can
+precede those input errors.
+The results record the fit-time policy in `unsupported_period_action`, `to_dict()`
+and `summary()`, including after aggregation or estimator reconfiguration.
+
+**All-eventually-treated panels (no never-treated group).** With the default
+`unsupported_period_action="drop"`, use
 `control_group="not_yet_treated"` — `"never_treated"` raises when no cohort-0
 units exist. Periods at which every unit is treated carry no identified
 ATT(g, t), so they are REMOVED from the estimation sample before the solve and
@@ -1652,11 +1668,14 @@ unaffected either way. Default `xtvar` (`demean_covariates=True`) is FULL
 RANK and fits cleanly.
 
 **Survey designs refuse row-dropping paths.** `survey_design=` combined with
-either comparison-support period filtering or unidentified-cohort exclusion
-raises `NotImplementedError` rather than deleting rows, because deletion would
+either comparison-support period filtering (`unsupported_period_action="drop"`)
+or unidentified-cohort exclusion raises `NotImplementedError` rather than deleting rows, because deletion would
 remove their PSUs and strata from the TSL variance. Restrict the frame yourself
-and re-fit. The refusals are conditional: survey fits that drop nothing are
-unaffected.
+and re-fit only after confirming every PSU and stratum survives that restriction.
+With `unsupported_period_action="error"`, unsupported periods instead raise
+`ValueError`, after the existing pre-filter checks, including survey-design
+validation. The refusals are conditional:
+survey fits that drop nothing are unaffected.
 
 **Alias:** `ETWFE`
 

diff_diff/guides/llms.txt

@@ -76,7 +76,7 @@ The site is organized into 5 sections, each with a landing page:
 - [EfficientDiD](https://diff-diff.readthedocs.io/en/stable/api/efficient_did.html): Chen, Sant'Anna & Xie (2025) efficient DiD with optimal weighting for tighter SEs
 - [TROP](https://diff-diff.readthedocs.io/en/stable/api/trop.html): Triply Robust Panel estimator (Athey et al. 2025) with nuclear norm factor adjustment (absorbing by default; `non_absorbing=True` for on/off treatment, method='local')
 - [StaggeredTripleDifference](https://diff-diff.readthedocs.io/en/stable/api/staggered.html#staggeredtripledifference): Ortiz-Villavicencio & Sant'Anna (2025) staggered DDD with group-time ATT. DEPRECATED in 3.9, removed in 4.0 - use `TripleDifference` with `first_treat=` (supplying the unit id, the calendar period column and `partition=`) - the same engine (`eligibility=` is `partition=` there; `control_group` takes the underscored values). Alias `SDDD` deprecated with it
-- [WooldridgeDiD](https://diff-diff.readthedocs.io/en/stable/api/wooldridge_etwfe.html): Wooldridge (2023, 2025) ETWFE — saturated OLS, logit/Poisson QMLE (ASF-based ATT). Alias: ETWFE
+- [WooldridgeDiD](https://diff-diff.readthedocs.io/en/stable/api/wooldridge_etwfe.html): Wooldridge (2023, 2025) ETWFE — saturated OLS, logit/Poisson QMLE (ASF-based ATT); `unsupported_period_action="error"` refuses comparison-support period filtering (default `"drop"`). Alias: ETWFE
 - [LPDiD](https://diff-diff.readthedocs.io/en/stable/api/lpdid.html): Dube, Girardi, Jorda & Taylor (2025) Local Projections DiD: per-horizon long-difference event study on clean controls (no negative weighting); variance- or equally-weighted ATT, premean differencing, pooled pre/post, fast. Absorbing by default; non-absorbing (reversible) treatment via `non_absorbing="first_entry"` (Eq. 12) or `"effect_stabilization"` (Eq. 13, window `L`). Complex-survey designs (pweight + stratified-PSU TSL SEs) on the default path via `fit(survey_design=...)`.
 - [ChangesInChanges](https://diff-diff.readthedocs.io/en/stable/api/changes_in_changes.html): Athey & Imbens (2006) nonlinear/distributional DiD for the 2x2 design: recovers the treated group's full counterfactual outcome distribution and quantile treatment effects (ATT + QTE grid) via the CDF transformation `F_10(F_00^{-1}(F_01(y)))`; invariant to monotone outcome transformations (unconditional fits; the covariate QR branch is not); bootstrap inference (panel or repeated cross-section resampling); point parity with R `qte::CiC()`, including its covariate branch (`covariates=` -> per-cell linear quantile regression, Melly-Santangelo-style conditional CiC). Continuous outcomes, numeric covariates. Alias `CiC`.
 - [QDiD](https://diff-diff.readthedocs.io/en/stable/api/changes_in_changes.html): **Deprecated 3.9, removed 4.0 - use `ChangesInChanges(method="qdid")`.** Athey & Imbens (2006) quantile DiD comparison estimator (additive quantile-by-quantile DiD, matching R `qte::QDiD()` including its covariate branch via `covariates=`); same bootstrap machinery as ChangesInChanges. The paper recommends CiC over QDiD (scale-dependent model with testable restrictions; a non-monotonicity warning fires when violated - unconditional fits only, the covariate-path counterfactual quantile curve is monotone by construction).

diff_diff/wooldridge.py

@@ -971,6 +971,21 @@ class WooldridgeDiD(BaseEstimator):
         always take precedence; the logit/poisson arms are knob-independent
         (survey df or normal theory — an explicitly non-default value warns
         at fit time). The default flips to ``"cluster"`` at v4.
+    unsupported_period_action : {"drop", "error"}, default "drop"
+        How to handle periods lacking the required comparison support.
+        ``"drop"`` removes those periods before estimation and warns;
+        ``"error"`` raises ``ValueError`` before removing them. Support
+        requires a positive-weight never-treated observation on OLS with
+        ``control_group="never_treated"``; other paths also admit observations
+        before ``g - anticipation``. This policy is independent of
+        ``rank_deficient_action`` and does not control unidentified-cohort
+        exclusion. With ``survey_design``, ``"drop"`` still raises
+        ``NotImplementedError`` if periods would be removed, because survey
+        domain estimation is not supported; ``"error"`` raises ``ValueError``
+        after the existing pre-filter configuration, cohort, and survey-design
+        checks. Later validation (including covariate columns, nonlinear
+        outcomes, and some explicit cluster columns) is not preflighted: an
+        unsupported-period refusal can precede those input errors.
     """
 
     def __init__(
@@ -993,6 +1008,7 @@ def __init__(
         conley_kernel: str = "bartlett",
         conley_lag_cutoff: Optional[int] = None,
         df_convention: str = "residual",
+        unsupported_period_action: str = "drop",
     ) -> None:
         self._validate_constructor_args(
             method=method,
@@ -1022,6 +1038,8 @@ def __init__(
         self.conley_kernel = conley_kernel
         self.conley_lag_cutoff = conley_lag_cutoff
         self.df_convention = df_convention
+        self._validate_unsupported_period_action(unsupported_period_action)
+        self.unsupported_period_action = unsupported_period_action
         # Track whether the user explicitly opted out of the "hc1" default.
         # The auto-cluster-at-unit default in `_fit_ols` is suppressed only
         # when the user explicitly opts into a one-way family (``hc2``,
@@ -1033,6 +1051,12 @@ def __init__(
         self.is_fitted_: bool = False
         self._results: Optional[WooldridgeDiDResults] = None
 
+    @staticmethod
+    def _validate_unsupported_period_action(value: str) -> None:
+        """Validate the period policy without coercing non-string values."""
+        if not isinstance(value, str) or value not in ("drop", "error"):
+            raise ValueError(f"unsupported_period_action must be 'drop' or 'error', got {value!r}")
+
     @staticmethod
     def _validate_constructor_args(
         *,
@@ -1152,6 +1176,7 @@ def fit(
         # mutated and passed the deprecated kwarg still sees the
         # FutureWarning before the raise.
         self.anticipation = validate_anticipation(self.anticipation)
+        self._validate_unsupported_period_action(self.unsupported_period_action)
 
         df = data.copy()
         df = _warn_and_fill_nan_cohort(df, cohort, stacklevel=2)
@@ -1501,6 +1526,18 @@ def fit(
         }
 
         if _unsupported_periods:
+            if self.unsupported_period_action == "error":
+                _n_unsupported = int(sample[time].isin(_unsupported_periods).sum())
+                _plabels = ", ".join(str(t) for t in _unsupported_periods)
+                raise ValueError(
+                    f"Period(s) {_plabels} have no eligible comparison group "
+                    f"and contain {_n_unsupported} of {len(sample)} observations. "
+                    "unsupported_period_action='error' refuses the fit before "
+                    "removing these unsupported periods. Use "
+                    "unsupported_period_action='drop' to permit automatic "
+                    "filtering (unavailable with survey_design), or supply data "
+                    "with the required comparison support."
+                )
             if survey_design is not None:
                 # Same naive-subsetting problem the unidentified-cohort path
                 # refuses below: deleting rows removes their PSUs and strata
@@ -2015,6 +2052,7 @@ def _build(frame: pd.DataFrame, w: Optional[np.ndarray]):
             and _pre_filter_unit_counts[g] > results._n_g_per_cohort[g]
         }
 
+        results.unsupported_period_action = self.unsupported_period_action
         self._results = results
         self.is_fitted_ = True
         return results