PolicyRecLab

A controlled research lab for off-policy evaluation, exploration, and policy-generated exposure bias in recommendation systems.

Research question: When can a target policy be reliably evaluated from data generated by a different logging policy?

PolicyRecLab studies a simple but easy-to-miss failure mode in offline recommendation research: an estimator can return a plausible number even when the logged data provide weak—or zero—information about the target policy.

The project therefore treats identification first, estimation second.

What this project shows

Setting	Headline result
Synthetic bandits	Zero support makes target-policy value not nonparametrically identified; weak overlap can make IPS extremely unstable even with correct propensities.
Open Bandit Dataset	IPS differed from independent Random-policy CTRs by about −1.0%, +2.5%, and −4.4%, while ESS fractions stayed below 0.2%.
Coat	A feature-aware exposure model removed about 63% of the observational/randomized mean-rating gap; a latent exposure model removed only about 3% and was rejected.
Yahoo R3	Rating-dependent Naive Bayes removed about 99% of the apparent gap, but the result is structurally tied to randomized calibration data.
Yahoo reliability	Across 50 randomized splits, 10% calibration was the first tested budget with consistently strong performance in this benchmark—not a universal threshold.

Close point-estimate agreement can coexist with weak overlap.

The main conclusion is not “use estimator X.” It is:

Never report an offline policy value without also reporting whether the logged data could reasonably identify it.

The reliability framework

PolicyRecLab separates three questions that are often mixed together:

1. Identification — does the logging process contain enough information to identify the target policy value under the stated assumptions?
2. Estimation — how do IPS, SNIPS, DM, DR, clipping, and reward/propensity models behave in finite samples?
3. Reliability — what support, weight, propensity, selection, and inference diagnostics should accompany the estimate?

A complete OPE report should state the estimand, logging process, propensity source, support condition, target-policy selection procedure, and inference method.

ESS is used here as a weight-concentration diagnostic, not as a literal inferential sample size.

Experimental program

The controlled synthetic sequence is:

simulation → exposure bias → support → IPS → weak overlap → clipping/SNIPS → exploration → DM → DR → policy selection → feedback loops → reliability reporting

Key retained findings:

• deterministic logging creates literal zero-propensity regions;
• unsupported target policies are not nonparametrically identified;
• IPS can be unbiased under correct support/propensities and still have extreme variance under weak overlap;
• clipping can reduce variance by introducing bias;
• SNIPS is not universally better than IPS;
• DM can be stable but biased under reward-model misspecification;
• DR does not repair zero support without additional assumptions;
• selecting and evaluating policies on the same OPE sample can induce winner's-curse behavior;
• stable weights or high ESS do not prove successful debiasing.

The synthetic simulator exposes exact finite-population policy value, so these effects can be measured directly against ground truth.

Real-data evidence

The real-data studies deliberately cover two different data-generating regimes.

Contextual-bandit benchmark: Open Bandit Dataset

OBD is a one-step production contextual-bandit benchmark with supplied logging propensities and an independent Random-policy deployment.

Across the all, men, and women campaigns, IPS estimates of Uniform Random from BTS logs differed from Random-policy CTRs by about −1.0%, +2.5%, and −4.4%. At the same time, maximum weights exceeded 12,000 and ESS fractions remained below 0.2%.

This is the clearest real-data example of the project's central point: reasonable point estimates do not imply comfortable overlap.

MNAR exposure benchmark: Coat

Coat is an MNAR rating-exposure benchmark, not a contextual-bandit log.

• randomized mean rating: 2.2289
• observational mean rating: 2.6115
• item-frequency weighting: about 18% gap reduction
• latent exposure model: about 3% gap reduction — rejected
• feature-aware exposure model: about 63% gap reduction
• released propensities: about 73% gap reduction

The failed latent model is intentionally preserved: nearly constant estimated propensities produced stable-looking weights but almost no useful debiasing.

Yahoo R3 MNAR calibration results

Yahoo R3 is also an MNAR rating-exposure benchmark, not a production contextual-bandit log.

• held-out randomized mean rating: 1.820
• observational mean rating: 2.892
• item-frequency weighting: 13.9% gap reduction
• rating-dependent Naive Bayes: about 99% apparent gap reduction

The Naive-Bayes result does not show that the observational data alone identify the MNAR exposure mechanism. Its success is structurally tied to the calibration subset, because randomized ratings supply information about the target rating distribution.

Across 50 randomized splits, the fraction of runs with absolute bias at most 0.02 increased from 72% at 5% calibration to 94% at 10% and 96% at 20%. For this benchmark, 10% randomized calibration is a defensible empirical operating point—not a universal threshold.

See docs/v1_3_7_yahoo_r3_results_synthesis.md.

Core estimand

For the one-step contextual-bandit experiments, the target policy value is

$$V(\pi_e) = \mathbb{E}_{X} \left[ \sum_{a \in \mathcal{A}(X)} \pi_e(a \mid X)\mu(X,a) \right]$$

The project focuses on when this value is identifiable and estimable from data generated by a different policy.

Reproduce the project

Start with a clean environment:

git clone https://github.com/bagheri365/PolicyRecLab.git
cd PolicyRecLab

python3 -m venv .venv
source .venv/bin/activate

python -m pip install --upgrade pip
python -m pip install -e .
pytest

The complete reproduction guide—including OBD, Coat, Yahoo R3, local-data rules, and experiment commands—is in docs/REPRODUCIBILITY.md.

Raw external datasets are not bundled with the repository. In particular, raw Yahoo R3 files should remain local and should not be committed or redistributed through this project.

Read next

For a fast path through the project:

1. Research synthesis: docs/FINAL_REPORT.md
2. Reproduction: docs/REPRODUCIBILITY.md
3. Milestone map: docs/DOCUMENTATION_INDEX.md
4. Yahoo synthesis: docs/v1_3_7_yahoo_r3_results_synthesis.md
5. Release audit: docs/RELEASE_CHECKLIST.md

Documentation and release status

The experimental program is frozen at v1.3.7. The first completed public research release is v1.0.0.

• docs/FINAL_REPORT.md — paper-style research report
• docs/DOCUMENTATION_INDEX.md — milestone map
• docs/DOCUMENTATION_AUDIT.md — documentation and scientific-boundary audit
• docs/REPRODUCIBILITY.md — fresh-clone and real-data reproduction guide
• docs/RELEASE_CHECKLIST.md — final release audit

New experiments are out of scope unless reproduction or documentation work reveals a correctness issue.

Philosophy

baseline → measurable failure → targeted intervention → evaluate → retain / reject

Change one mechanism, measure it, and preserve the evidence—including failed, rejected, and invalidated experiments.