Build-generated collection manifest + pluggable LibrarySource (authoritative sizing & prerelease)

[Rick-Wilson]

Summary

Give the backend an authoritative, storage-agnostic view of each collection's shape (board rosters, sizing, release status) via a build-generated manifest that both Baker Bridge and Practice-Bidding-Scenarios publish in one shared format, consumed behind a pluggable LibrarySource abstraction.

Design docs (this issue tracks their implementation):

• ADR: documentation/adr/0002-collection-manifest-and-library-source.md (amends the Collection Producer Contract with a new R5)
• Design: documentation/design/collection-manifest-and-library-source.md

Why

Today the backend has no authoritative board count and no server-side view of release status:

• Lesson size is inferred as COUNT(DISTINCT deal_number) across all users (lesson_mastery.rs) — so a lesson's "size" is only as big as the union of boards anyone has played. Denominators understate until coverage fills in.
• prerelease is client-asserted: the browser computes prerelease = deal.stable !== true (observationSchema.js) and the backend trusts it. The producer's stable flag lives in the PBN, but nothing server-side reads the PBN.
• Three parallel, non-authoritative sizing sources exist: static dealCount table (bakerBridgeTaxonomy.js), runtime PBN regex parsing (useBoardMastery.js), and backend COUNT(DISTINCT).

The frontend already fetches a whole-collection manifest for PBS (fetchScenarioManifest, carries generatedAtCommit) — but it has no sizing, and Baker Bridge has no equivalent (uses a differently-shaped toc.json).

Scope

Content producers

• Agree manifest schema v2 (adds a lessons board roster) — Rick + David
• PBS: add the lessons roster to David's manifest generator
• Baker Bridge: new manifest generator in CSVtoPBN (companion to the R3 board-version-token stamping step)

Backend (Bridge-Classroom-api)

• LibrarySource trait + GitHubRawSource impl (conditional GET / ETag / generatedAtCommit)
• CollectionRegistry config (collection → source + location); fold in reports.rs::repo_for_collection
• In-memory manifest cache keyed by (collection_id, version_token)
• Change-detection loop: lazy-on-read TTL + optional background refresh (webhook deferred)
• Cut lesson_mastery.rs denominators over to the roster (verify vs current values before removing COUNT(DISTINCT))
• Make prerelease server-authoritative: reconcile observations.prerelease against the roster in student_summary.rs stats

Frontend

• Delete runtime PBN parsing for counts (useBoardMastery.js) and the static dealCount table (bakerBridgeTaxonomy.js) once the backend is authoritative
• (later) reconcile COLLECTIONS[] location fields against the backend registry (single source)

Key decisions (see ADR)

• Backend pulls, frontend does not push. Client-pushed sizing would re-send per session and re-introduce the client-trust problem. Backend fetches the manifest itself.
• Collection location is backend-owned config. The backend must know where to fetch regardless; consolidate the reports.rs hardcode + frontend duplication into one registry. Client keeps stamping collection_id per observation (cheap, self-contained), but the backend stops trusting the client for sizing/release status.
• Reconciliation, not rejection. An observation for a board not in the roster is accepted + flagged, never rejected — ADR-0001 observations stay self-contained; history is never mutated.
• Pluggable storage. GitHub-raw is impl Improve DB structure so we don't need to pull full history to show summary views #1; GoogleDocsSource / ObjectStoreSource / LocalFileSource drop in behind the trait without touching routes.
• No GitHub "watch" needed. generatedAtCommit + If-None-Match conditional GETs; a 304 is free. Webhook is an optional later accelerator.

Open questions

• Registry format (checked-in TOML/JSON vs. backend serving it to the frontend too)
• Keep Baker toc.json alongside the manifest, or subsume it?
• Poll TTL / background interval values
• Skill-path source of truth (manifest vs frontend taxonomy) once the manifest exists

[Rick-Wilson]

ADR-0002 ratified & merged (#90 / #91). Contract decision + producer R5 are now docs on main.

Producer-side work split out to the collection repos:

• PBS manifest v2 roster → Manifest v2: add authoritative board roster (producer contract R5) Practice-Bidding-Scenarios#179
• Baker Bridge manifest generator → Emit a build-generated manifest matching PBS schema v2 (producer contract R5) Baker-Bridge#8

This issue now tracks the consumer (backend + frontend) side only — the LibrarySource trait + GitHubRawSource, CollectionRegistry (folding in reports.rs::repo_for_collection), manifest cache + change-detection loop, cutting lesson_mastery denominators over to the roster, making prerelease server-authoritative, and deleting the frontend's runtime PBN parsing + static dealCount. The producer checkboxes above are now owned by the two repo issues.

[Rick-Wilson]

Consumer sub-task filed: #94 — frontend manifest-join (source board_version_token from the manifest roster instead of the PBN tag). This is the closing path for the PBS #175 descope and the null-token window, and reuses the same roster parse the mastery-denominator work here needs.