Interactive, privacy-first ML platform —
app.dominicdapice.com/mlA personal, zero-friction Data/ML hub with its own visual identity and a visible MLOps layer.
What we are building. A complete web application where a visitor can:
- Drag in a CSV file (or pick a demo dataset),
- Explore their data (types, distributions, missing values),
- Choose a target column (or let it be detected),
- Automatically train several models entirely in their browser,
- Read a leaderboard, metrics, charts, feature importance and a natural-language interpretation,
- Save their projects locally, export models and reports, share their results without sharing their data.
Critical point retained: we ship a complete application, not just the shell. The shell (layout, navigation, theme, /ml, /data, /ai routes) is the deliverable of Sprint 0 only. The heart of the project — and the bulk of the plan below — is the end-to-end working ML engine: parsing → profiling → preprocessing → multi-model training → evaluation → visualization → export. Every sprint ends with a feature a real visitor can use, not a mockup.
Structuring constraints:
- Domain:
app.dominicdapice.com, path-based routing (/mlfirst,/dataand/ailater). - Infra: GitHub (code + Actions CI/CD) + Cloudflare (DNS, SSL, hosting).
- Privacy-first: no user data ever leaves the browser.
- Bilingual French / English from the very first version: visible language switcher, browser-language detection, remembered preference. Every UI string goes through the i18n layer — no hard-coded strings.
- Two color themes, user's choice: light and dark, visible toggle in the header, remembered preference (default = system preference), WCAG contrast verified in both themes (ECharts charts included).
- Responsive, accessible (WCAG), clean code (ESLint/Prettier), open-source.
Note: this synthesis is based on observing a zero-friction ML lab on the market, taken as the journey reference, plus the observations provided in the brief.
User journey (the "flow" to match, then beat):
| Step | Reference behavior |
|---|---|
| Entry | Drag & drop a CSV (or tabular format), read locally, nothing is sent to a server |
| Target | The user picks the column to predict, or ML Lab suggests it; suggestions of columns to keep/exclude |
| Task | Automatic classification-vs-regression detection from the target column |
| Training | A small set of models trained client-side, with a held-out test set |
| Results | Model leaderboard, metrics compared to a naive baseline, charts, top features, and a natural-language "plain read" |
| Friction | Zero: no account, no setup, no code |
Design/UX (the spirit to capture): a clean, professional SaaS interface — one task per screen, guided progression (upload → target → train → results), plenty of white space, clear typographic hierarchy, results explained for non-experts.
What makes the product strong (and what we keep): zero friction, a verifiable privacy promise, the naive baseline as an honest point of comparison, the natural-language interpretation.
What we will do better (detailed in section D): visible MLOps observability, reproducibility ("pipeline as code"), persistent projects, data-free sharing, deeper interpretability, dark/light theme, offline PWA.
| Layer | Recommended choice | Alternatives set aside |
|---|---|---|
| Framework | Vite + React 19 + strict TypeScript (static SPA) | Next.js (SSR is useless here: all compute is client-side; Next's static export adds complexity for no benefit), Nuxt/SvelteKit (same reason) |
| Routing | React Router v7 (library mode, per-route code-splitting) | TanStack Router (excellent but more niche) |
| UI | Tailwind CSS v4 + shadcn/ui (components copied into the repo, customizable) + lucide-react | Material UI (visual identity too "Google", heavier bundle) |
| State | Zustand (light, testable) | Redux (oversized) |
| Charts | Apache ECharts (canvas, fast on 100k+ points, tree-shakable, native zoom/filter/export) | Plotly.js (~4 MB, too heavy for the perf budget), pure D3 (high dev cost) |
| Parsing | Papa Parse (streaming CSV, Web Worker support) | SheetJS (added in V2 for .xlsx) |
| Classical ML | ml.js ecosystem (ml-cart, ml-random-forest, ml-knn, ml-naivebayes, ml-regression) + hand-written TS implementations (logistic regression, histogram GBDT) | scikit-learn on a server (breaks the privacy promise) |
| Neural networks | TensorFlow.js (MLP, WebGPU backend with WASM fallback) — V2 | — |
| Pre-trained model inference | ONNX Runtime Web (/ai/vision module) — V3 |
— |
| i18n | react-i18next (FR/EN, browser detection, lazy-loaded resources) | home-grown solutions (reinventing the wheel) |
| Persistence | Dexie.js (IndexedDB): projects, runs, models | localStorage (size limits), backend (privacy) |
| Sharing | lz-string: results compressed into the URL fragment (#…) |
Sharing backend (optional V3) |
| Hosting | Cloudflare Pages (static + CDN) | Workers Sites (pointless without server logic) |
- Static SPA over full-stack. All the value (parsing, training, visualization) runs on the client: an application server would only add cost, attack surface, and a weakened privacy promise. Infra parallel: this is the equivalent of a site served by Azure Static Web Apps + CDN — zero servers to patch, trivial scaling, near-zero cost.
- Vite + React. Fast builds, the richest JS ML/dataviz ecosystem, and the most legible skill on a portfolio. Strict TypeScript plays the role Terraform validation plays: errors are caught at "plan" time, not at "apply" time.
- Web Workers for training. The UI must never freeze during a fit. Each training run goes to a dedicated worker (via Comlink) that publishes progress events. Parallel: this is your job queue — the worker is a build agent, the UI is the orchestrator streaming the logs live.
- ECharts over Plotly. The performance budget (Lighthouse ≥ 95) is a stated goal; Plotly alone would blow it. ECharts provides the zoom, brush, filtering and PNG export the brief asks for, at ~1/5 of the weight with selective imports.
- shadcn/ui over a component library. The components live in our repo: our own visual identity (a brief requirement), no external style dependency, and a showcase of readable code.
┌─────────────────────────────────────────────┐
│ THE USER'S BROWSER │
│ (all the data stays here) │
│ │
│ ┌───────────────┐ ┌────────────────────┐ │
│ │ React UI │◄──┤ Web Workers │ │
│ │ /ml /data /ai│ │ · parsing (Papa) │ │
│ │ ECharts │ │ · preprocessing │ │
│ │ Zustand │ │ · training │ │
│ └──────┬────────┘ │ (ml.js / TF.js) │ │
│ │ └────────────────────┘ │
│ ┌──────▼────────┐ ┌────────────────────┐ │
│ │ IndexedDB │ │ URL fragment #… │ │
│ │ (Dexie) │ │ data-free sharing │ │
│ │ projects/runs │ │ (never sent to │ │
│ └───────────────┘ │ the server) │ │
└─────────┬────────────┴────────────────────┴─┘
│ HTTPS (static assets only)
┌────────────────▼───────────────────┐
│ CLOUDFLARE │
│ DNS: app.dominicdapice.com (CNAME)│
│ Universal SSL · CDN · _headers CSP│
│ Pages (static) · Workers (V3: │
│ API proxy for /ai/chat) │
└────────────────▲───────────────────┘
│ deployment (wrangler)
┌────────────────┴───────────────────┐
│ GITHUB │
│ LabML repo (open-source) │
│ Actions: lint→type→test→build→ │
│ e2e→deploy (+ per-PR previews) │
└────────────────────────────────────┘
Key privacy point: the share link encodes the metrics/charts (never the data) into the URL fragment — the part after # is never sent to the server by the browser. The promise "your data never leaves your machine" stays true even while sharing.
LabML/
├── .github/workflows/ # ci.yml, deploy.yml, lighthouse.yml
├── docs/ # ARCHITECTURE.md, adr/, user guide
├── public/
│ ├── datasets/ # demo CSVs (iris, titanic, housing…)
│ └── _headers, _redirects # CSP + Cloudflare SPA fallback
├── src/
│ ├── app/ # bootstrap, router, layout, theme
│ ├── components/ui/ # design system (customized shadcn/ui)
│ ├── features/
│ │ ├── ml/ # THE HEART OF THE PRODUCT
│ │ │ ├── data/ # parsing, type inference, profiling
│ │ │ ├── pipeline/ # preprocessing, split, run config
│ │ │ ├── models/ # one module per model (shared interface)
│ │ │ ├── metrics/ # accuracy, F1, AUC, RMSE… (golden-tested)
│ │ │ ├── explain/ # importance, PDP, NL interpretation
│ │ │ ├── workers/ # training workers (Comlink)
│ │ │ ├── projects/ # Dexie persistence, export, sharing
│ │ │ └── pages/ # screens of the /ml journey
│ │ ├── data/ # V3 placeholder (data pipelines)
│ │ └── ai/ # V2/V3 placeholder (chat, vision)
│ └── lib/ # shared utilities
├── tests/
│ ├── e2e/ # Playwright
│ └── golden/ # scikit-learn reference values
└── wrangler.toml # Cloudflare config (the .tf equivalent)
Each section (/ml, /data, /ai) is a lazily-loaded feature folder (per-route code-splitting): adding /ai/vision later touches neither the shell nor /ml.
- Vite + React + strict TS scaffold, ESLint (flat config) + Prettier + Husky/lint-staged.
- Design system: tokens (colors, type, spacing), two light/dark themes with a persisted user toggle (default = system preference), base components.
- i18n foundation (react-i18next): FR + EN from the very first screen, language switcher, browser detection, persisted preference — i18n lands in Sprint 0 because retrofitting it onto an existing app costs 10× more.
- Shell: layout, navigation,
/,/ml,/data(coming soon),/ai(coming soon) pages, 404 page. - Full CI/CD (section C) + the
app.dominicdapice.comdomain live. - Definition of done: the app is deployed, Lighthouse ≥ 95, the CI pipeline blocks on lint/type/test.
- Drag & drop CSV upload (streaming Papa Parse in a worker, up to ~500 MB).
- Built-in demo datasets (Iris, Titanic, California Housing) → one-click trial, zero friction.
- Per-column type inference (numeric, categorical, boolean, date, text, ID).
- Data profile: per column — distribution (histogram/bars), % missing, cardinality, min/max/mean/median; virtualized tabular preview.
- Target selection + automatic task detection (binary/multi-class classification vs regression).
- Smart suggestions: exclusion of ID/constant/near-empty columns, target-leakage warning (a column too correlated with the target).
- DoD: a visitor loads a CSV and understands their data without installing anything.
- Declarative preprocessing pipeline: imputation (median/mode), encoding (one-hot / ordinal depending on cardinality), standardization, all fitted on the training split only (no leakage).
- Stratified train/test split (80/20, fixed reproducible seed).
- Model zoo v1 behind a shared
Trainableinterface:- Naive baseline (majority class / mean) — the honest point of comparison,
- Linear / logistic regression (hand-written TS, vectorized gradient descent),
- k-NN, Gaussian Naive Bayes,
- Decision tree (ml-cart), Random Forest (ml-random-forest).
- Execution in Web Workers with live progress (current model, %, elapsed time), cancellable.
- Live leaderboard: rows appear as they finish, sorted by the main metric, delta vs baseline highlighted.
- Metrics: accuracy, precision/recall, F1, ROC-AUC, log-loss (classification); RMSE, MAE, R² (regression).
- DoD: the full upload → target → train → leaderboard journey, reproducible (same seed ⇒ same results).
- Per-model visualizations: interactive confusion matrix, ROC/PR curves, predicted-vs-actual and residuals (regression) — zoom, filter, PNG export (ECharts).
- Feature importance: permutation importance (model-agnostic) + impurity-based importance for trees.
- Natural-language interpretation generated by rules (no external API): "The Random Forest beats the baseline by 18 points. The 3 most decisive variables are…".
- What-if prediction: a form pre-filled with one row; the user edits values and watches the prediction change live.
- DoD: the results page tells a complete story to a non-expert.
- Projects (Dexie/IndexedDB): run history, rename, delete, side-by-side comparison of two runs.
- Exports: model (reloadable JSON), predictions (CSV), self-contained HTML report (printable to PDF via CSS print).
- Share link: metrics + charts compressed (lz-string) into the URL fragment; read-only page.
- DoD: a returning visitor finds their projects; a shared link opens without the original data.
- WCAG AA accessibility (keyboard navigation, aria, contrast verified in both themes, axe-core in CI).
- Offline PWA (vite-plugin-pwa): the ultimate demo of the privacy promise — turn off the Wi-Fi, everything still works.
- Lighthouse budgets in CI, full FR/EN translation review (including the generated natural-language interpretations, produced in both languages), "How it works / Privacy" page.
- V2: TensorFlow.js MLP (WebGPU→WASM), hand-written TS histogram GBDT (LightGBM spirit), hyperparameter search (time-budgeted random search), approximate SHAP (sampled Kernel SHAP), PDP/ICE, .xlsx/.parquet import.
- V3:
/ai/vision(ONNX Runtime Web + pre-trained models like MobileNet),/ai/chat(LLM via a Cloudflare Worker proxy — only the questions and aggregated statistics leave, never the raw data, with explicit consent), the/datamodule, ONNX export, GitHub OAuth auth if features ever justify it.
Two workflows (a direct parallel with your Azure DevOps pipelines):
ci.yml (on PRs and main)
lint (ESLint + Prettier check)
→ typecheck (tsc --noEmit)
→ unit + golden tests (Vitest, coverage)
→ build (Vite)
→ e2e (Playwright, Chromium browser)
→ lighthouse-ci (perf/a11y budgets, non-blocking at first)
deploy.yml (after green CI)
PR → wrangler pages deploy --branch=<pr> ⇒ per-PR preview URL
main → wrangler pages deploy ⇒ production
- Deployment via
cloudflare/wrangler-actionwithCLOUDFLARE_API_TOKEN+CLOUDFLARE_ACCOUNT_IDas GitHub secrets (the equivalent of Azure DevOps variable groups; token scoped to the single Pages project, least privilege). - Per-PR previews = your ephemeral environments: every PR gets its own test URL.
- V2: CodeQL + Dependabot/Renovate — a DevSecOps showcase.
- Pages:
labmlproject, connected to the repo (deployment driven by Actions to keep CI in control — the "plan/apply" stays in GitHub). - DNS:
app.dominicdapice.comas aCNAMEto<project>.pages.dev, proxy enabled (orange cloud) ⇒ automatic universal SSL, CDN, HTTP/3. The apex domain stays on GitHub Pages, no interference. - Path-based routing: the SPA router handles
/ml,/data,/ai— on the Cloudflare side, a simple_redirectsfallback (/* /index.html 200). No Workers needed for the MVP: a routing Worker would only make sense if/aiever became a separate app. _headers: strict CSP and security headers (section E), served by the CDN.
- Built-in run observability. For every model, the leaderboard shows: training time, p50/p95 inference latency (measured on the test set), estimated memory, compute backend (WASM/WebGPU). A lay visitor sees scores; a recruiter sees a Grafana/App Insights reflex applied to ML. (Sprint 2–3, low marginal cost.)
- Pipeline as code + reproducibility. Every run is defined by a declarative config (JSON: seed, split, preprocessing, models, hyperparameters), viewable via a "View pipeline" button, re-runnable identically, and exportable as an equivalent scikit-learn Python script. This is Terraform applied to ML: the run is the
applyof a versioned plan. (Config from Sprint 2; button + Python export in Sprint 4/V2.) - Local model registry with lineage. Saved models carry a version, a stage (dev/staging/prod), their original config and a hash of the data schema; diff comparison between two runs and a schema-drift alert when a new CSV in the same project no longer matches. MLflow-like, 100% in IndexedDB. (Sprint 4 for the base, V2 for drift.)
| Requested improvement | Priority | Where |
|---|---|---|
| Interactive charts (zoom, filter, export) | MVP | Sprints 1–3 (ECharts) |
| Project management (local history) | MVP | Sprint 4 (IndexedDB) |
| Data-free sharing | MVP | Sprint 4 (URL fragment) |
| Model export (JSON) + reports (HTML/PDF) | MVP | Sprint 4 |
| Light/dark theme, user's choice | MVP (hard req) | Sprint 0 |
| Bilingual FR/EN interface | MVP (hard req) | Sprints 0 → 5 |
| Smooth animations, WCAG | MVP | Sprints 0 + 5 |
| Simplified interpretability (importance, plain read) | MVP | Sprint 3 |
| Neural networks (TensorFlow.js) | V2 | MLP after the classical zoo |
| Boosted models (XGBoost/LightGBM spirit) | V2 | hand-written TS GBDT; WASM port to eval |
| Approximate SHAP/LIME, PDP/ICE | V2 | after permutation importance |
| AutoML + hyperparameters | V2 | budgeted random search |
| ONNX / PMML export | V3 | non-trivial in the browser, revisit |
| AI chat over the data | V3 | Worker proxy + explicit consent |
| Computer vision | V3 | ONNX Runtime Web, /ai/vision |
| Authentication | V3 | only if a real need emerges |
Prioritization logic: the MVP must beat the reference on its own turf (tabular journey + trust + persistence + sharing) before widening the surface. Every V2/V3 item is an independent brick thanks to the feature-folder split.
- The "data never leaves the browser" guarantee, verifiable:
- Strict CSP via
_headers:default-src 'self'— the browser forbids any network call to third parties; fonts and assets self-hosted. The promise is not marketing copy, it is observable in the Network tab (and the Privacy page will say exactly that). - No telemetry by default; at most, Cloudflare Web Analytics (cookie-less, aggregated) — and only if you decide so.
- Offline PWA = proof by usage.
- Strict CSP via
- Headers:
Strict-Transport-Security,X-Content-Type-Options,Referrer-Policy,Permissions-Policy; COOP/COEP only if we enable WASM multithreading (SharedArrayBuffer) in V2. - API keys: none in the client, ever. The future
/ai/chatwill go through a Cloudflare Worker proxy that holds the key (Wrangler secret), applies rate-limiting, and only relays consented aggregates. - Supply chain: committed lockfile, Dependabot/Renovate,
npm auditin CI, CodeQL (V2), minimal GitHub Actions permissions (permissions: contents: readby default).
- Unit (Vitest): critical components + the entire
metrics/,pipeline/,models/layer. - Golden tests — the centerpiece for a hand-written ML engine: a Python script (run outside CI, fixtures committed) computes scikit-learn reference values (metrics, predictions of simple fixed-seed models); Vitest checks that our TS implementation reproduces those values within tolerance. This is our scientific non-regression contract.
- Property-based (fast-check): CSV parsing (encodings, quotes, malformed lines), metric invariants (bounds, symmetries).
- E2E (Playwright): the full journey — load Titanic, pick
survived, train, check the leaderboard, export the report, reload the page and find the project again. - Performance: Lighthouse CI with budgets (perf ≥ 95, a11y ≥ 95, initial bundle < 200 KB gz — ML libs lazy-loaded at the first training).
- A11y: axe-core integrated into the e2e suite.
- README: pitch, screenshot, CI badges, 3-command quickstart.
- docs/ARCHITECTURE.md + ADRs (
docs/adr/): every structuring choice recorded (framework, ECharts, ML engine…) — the RFC/design-doc reflex teams expect from a tech lead. - Built-in user guide inside the app (a "How it works" page) rather than an external wiki.
- Maintenance: Renovate (weekly grouped updates), CHANGELOG with semantic versioning, light monitoring (Cloudflare Analytics + an error boundary with an error report the user can copy manually — no automatic sending, consistent with privacy).
Decisions validated on 20/08/2026:
| # | Decision | Validated choice |
|---|---|---|
| 1 | Framework | Vite + React + TypeScript (static SPA) |
| 2 | UI / design system | Tailwind CSS v4 + shadcn/ui |
| 3 | ML execution (MVP) | 100% in the browser (Azure reserved for V3 extensions) |
| 4 | Deployment | GitHub Actions + wrangler (per-PR previews) |
| 5 | Languages | Bilingual FR/EN from Sprint 0 (addendum) |
| 6 | Themes | Light + dark, user's choice (addendum) |
| 7 | Palette | Deep teal + copper accent — no yellow/amber (an identity deliberately distinct from the reference) |
Still-open questions (non-blocking for Sprint 0):
- Default language when the browser reports neither FR nor EN (recommended: English, for the portfolio's international reach).
- Cloudflare access: create the Pages project + API token on your side (step-by-step guide provided in Sprint 0), or validate first on
*.pages.devbefore wiring the DNS?
- You: validate/amend the answers to questions H1–H6.
- Me: Sprint 0 — full scaffold + design system + shell + CI/CD (dedicated PR, with preview).
- You: create the Cloudflare Pages project + token (guide provided), add the 2 GitHub secrets.
- Me: Sprints 1 → 2 right after — the app becomes a real ML Lab (upload → profiling → training → leaderboard), then Sprints 3 → 5.
- Joint review at the end of every sprint on the preview URL.
Recap: everything this plan promised is delivered and verified in production — S0–S5 (complete MVP), V2 (GBDT + MLP + PDP + Excel), V3 (ONNX vision), V4 (Data Studio), V5 (hyperparameter search + Shapley), V6 (FR/EN data assistant). 12 PRs merged, 152 unit tests, 32 e2e, WCAG AA, offline PWA.
Cap 2 prioritizes what raises the product's value the most while honoring the invariants (100% in the browser, strict CSP with no third parties, seeded determinism, FR/EN, WCAG AA, honest evaluation):
| Wave | Content | Why |
|---|---|---|
| V7 | Unsupervised exploration in the ML Lab: hand-written k-means (seeded k-means++ init, k ∈ 2–5 chosen by silhouette), hand-written 2D PCA projection (power iteration), plain-language group profiles, scatter plot with colors and shapes (palette validated for color blindness by the design-system validator) | Fills the real gap: today the lab requires a target; many datasets are explored first without one |
| V8 | Time series: date + numeric target detection → trend/season decomposition, hand-written Holt-Winters forecasting, rolling-origin backtest | Opens up an entire class of problems |
| V9 | Performance & comfort: /ml Lighthouse budget ≥ 0.90 (preloads, splitting), PWA update toast, webcam for vision (Permissions-Policy to open) | Perceived quality and scores |
| V10 | Data Studio 2: importable recipe replayable on a new file, per-column forced types, derived columns | Completes the reproducibility loop |
| — | Generative chat (optional, outside the cap): requires a server proxy (Cloudflare Worker) + a key provided by the owner — never a key in the browser; mandatory consent screen | A product decision to make separately |
Cap 2 recap: V7 (unsupervised exploration), V8 (time series), V9 (speed & comfort, /ml 0.77 → 0.86 measured), V10 (replayable recipes, forced types, derived dates) — delivered and verified in production. 16 PRs merged, 177 unit tests, 36 e2e.
| Wave | Content | Why |
|---|---|---|
| V11 — delivered | Data drift in the Data Studio: a reference file, a file to compare → schema differences (columns added/removed/retyped), PSI per column (quantile bins from the reference, thresholds 0.1/0.25), new/vanished categories, missing-rate gaps, overall verdict — with a deliberately drifted demo (cafe-sales-june.csv) |
The MLOps gesture par excellence: checking that a new batch looks like what the model learned on |
| V12 — pending | Consented generative chat: a Cloudflare Pages Function (same repo) proxying the Anthropic API — the key lives in a Cloudflare secret, never in the browser; the LLM translates the question into an intent executed locally by the V6 engine (only the question and the column schema leave, never the data); explicit consent screen; clean degradation if the secret is not configured | Owner's product decision (21/08/2026): postponed for now. Confirmed 09/10/2026, after Cap 7: not scheduled. |
| V13 — delivered | Complete runs: tuning, latest Shapley explanation, exploration and forecast attached to the run record — IndexedDB history (with chips), stored-run page, HTML report and v2 share links (subsampled scatter plots in the URL; v1 links remain decodable) | The V5–V8 artifacts did not survive the run |
| V14 — delivered | Generalized prerendering: static shells for all six sections (the V9 approach extended — inlined CSS, Latin fonts as data:, per-route preloaded façade, header template), Lighthouse /ml 0.86 → 0.99 and /data 1.0 under real throttling (3-run medians); the root stays the SPA fallback (accepted) | The last Lighthouse gap |
Cap 3 recap: V11 (drift), V13 (complete runs), V14 (generalized prerendering, /ml 0.99 under real throttling) — delivered and verified in production. V12 (consented generative chat) remains pending a product decision. Cap 4's guiding thread: the next gesture — using the model over time, and understanding it more finely.
| Wave | Content | Why |
|---|---|---|
| V15 — delivered | Score a new batch: after a run, drop a new file → the inspected model scores it in the browser (exportable predictions, all columns preserved); if the target is present, honest test-vs-batch comparison (unknown labels excluded and counted); schema validated, drifted demo iris-field.csv, score attached to the run (history/report/share) |
The complete MLOps loop: V11 says "the inputs moved", V15 says "does the model still hold" |
| V16 — delivered | Imbalance & thresholds: precision-recall curve (AP, chance line drawn), adjustable decision threshold priced by a cost matrix (false alarm vs missed case, one-click optimum), calibration curve (Brier), imbalanced demo fraud.csv; the chosen threshold joins the run. Accepted descope: class weighting at training time is set aside — the cost-priced threshold gives the same control at the decision level, without touching the eight hand-written models |
Real datasets are imbalanced; accuracy lies there |
| V17 — delivered | Data Studio 3: joins & anomalies: left join of a second file on a shared key (exact match after trim — a dirty key becomes a named orphan, never silence; match rate, duplicates, unused rows; the joined result becomes THE dataset), and multivariate anomalies via a hand-written seeded isolation forest (100 trees, exact c(n)) as a step of the replayable recipe (threshold 0.6). Accepted nuance: the join is an ingestion gesture, not a recipe step — replaying a recipe only requires a single file | Real data prep starts by crossing two files; multivariate anomalies see what Tukey misses |
| V18 — delivered | Per-segment analysis: after a run, the test set is sliced by every categorical column — including those excluded from the features, where proxy effects hide — and the inspected model's metric (accuracy or RMSE) is recomputed per slice, gap vs global signed and sorted worst-first. Slices < 8 rows set aside and counted, columns ranked by amplitude (cap 6×8), analysis attached to the run (history/report/share). On titanic it points at deck C, Cherbourg — and the alive column, sliced despite being excluded from training |
"Where does my model fail?" — an honest gateway to fairness |
Cap 4 closed (21/08/2026): V15, V16, V17 and V18 delivered and verified. Only V12 remains pending a product decision. The "Where the build stands" section of /ml was removed the same day (owner request) — the wave history lives here and in the README.
Cap 4 recap: V15 (batch scoring), V16 (imbalance & thresholds), V17 (joins & anomalies), V18 (per-segment analysis) — delivered and verified in production on 21/08/2026. Cap 5's guiding thread: the complete project — it survives, it measures itself, it compares itself, it gets reused.
| Wave | Content | Why |
|---|---|---|
| V19 — delivered | Persistent projects: the dataset joins the project, opt-in ("keep in this browser") — lz-string-compressed CSV in IndexedDB, explicit 50 MB budget (named refusal with the numbers, never a silent cut), saved list (reopen/forget) under the history, runs linked to the saved dataset ("reopen this run's data"), identical retraining (seed 42). Nothing in the share links | A refresh erased everything; "projects" are only real if they survive |
| V20 — delivered | Honest uncertainty: seeded bootstrap of the test set (1,000 resamples shared across models — paired comparisons) → percentile 95% CI on every leaderboard model's main metric (whiskers on a shared scale), plain-language paired winner-vs-baseline verdict ("the gap survives resampling — probably real" / "the interval crosses zero — possibly noise"), analysis attached to the run (history/report/share). Limits owned and displayed: the CI measures sensitivity to the test draw, not training variance; test set < 8 rows → refusal, no theater | 0.82 on 178 rows is not 0.82; say what the number does not say |
| V21 — delivered | Compare two runs: check two runs in the history → side-by-side diff on /ml/compare — features added/removed as ± badges, every model's metric in an A/B/Δ table (signed colors), plain-language read of the best model's movement, and a cross-run verdict when both runs carry V20 CIs (disjoint → the gap exceeds both uncertainties; overlapping → possibly noise). Honesty: different targets or task families → metric deltas refused (the config diff remains); owned note: two runs are never paired — an indication, not a test | "Did my cleaning help?" — the central iterative gesture of ML |
| V22 — delivered | The model comes back: export as format v2 — the JSON embeds the fitted pipeline (imputation/encoding/standardization), the target, the classes and the exporting run's test metrics as an honest reference. Re-imported on /ml, LabML rebuilds the EXACT predictor (every exportable family round-trips byte-identically — GBDT with its bin edges, MLP, trees/forests via ml.js) and scores any CSV without retraining; named refusals (unknown app, v1 format, missing columns, unknown family). k-NN always refuses to export: its "parameters" are the training data | Closes the last loop: train today, come back in a month, score |
Cap 5 closed (21/08/2026): V19, V20, V21 and V22 delivered.
Set aside for now (possible Cap 6): Vision 2 — a stronger model than a 2012 SqueezeNet, or real face/object detection, still 100% in the browser (owner request, 21/08/2026: portraits have no ImageNet class and the model answers off-target); text columns (hand-written TF-IDF), learning curves, scaling to 1M rows (typed arrays), multiclass thresholds. V12 remains pending a product decision.
Cap 5 recap: V19 (persistent projects), V20 (honest uncertainty), V21 (run comparison), V22 (model export/import) — delivered and verified in production on 21/08/2026. Cap 6's guiding thread: the lab meets the real world — real photos, real text, real file sizes, and the question every data budget asks.
| Wave | Content | Why |
|---|---|---|
| V23 — delivered | Vision 2: SqueezeNet (2012) retired for three self-hosted ONNX models — EfficientNet-Lite4 int8 classification (1,000 ImageNet classes, 77.6% top-1), YOLOX-Nano object detection (80 COCO classes; the stronger-but-AGPL YOLOs were ruled out, Apache-2.0 kept) and UltraFace RFB-320 face detection — boxes drawn on the image, FR/EN class names, plain-language counts ("1 person · 1 face"). Box decoding (grids/strides, exp, IoU, per-class NMS) is hand-written and unit-tested; letterboxed inputs (aspect squashing measurably mislocated face boxes); named thresholds (objects 35%; faces 0.9 — real faces score ≥ 0.95, measured false positives top out at 0.85); ~19 MB total, runtime-cached, offline after first use; verified on real photos (NASA portrait → 1 person + 1 face; German Shepherd → dog + breed at 99.9%) | Owner request (21/08/2026): portraits have no ImageNet class, so the old model answered off-target — and the detector must recognize a whole range of things, not just faces |
| V24 — delivered | Text columns: free text stops being skipped and enters the pipeline as a hand-written TF-IDF block — accent-folding bilingual tokenizer, merged FR/EN stop words, vocabulary capped at 256 terms ranked by document frequency (ties alphabetical, terms seen in a single training document dropped), smoothed IDF, L2-normalized vectors, fitted on the training split only. Features are named column:word, so importance, Shapley and the report speak in words; encodedBlocks now measures a text block by its real width (counting it as one column silently shifted every block after it). Explanations gained signed word effects by occlusion — erase one word from the reviews containing it and average the shift of the answer — because permutation is blind to a redundant vocabulary, and multiclass is refused rather than faked. Export bumped to format v3 (v2 files still import). Demo reviews.csv: 240 bilingual orders where the text carries the signal — baseline 0.52 → 0.92, review top of the importance chart, fast/excellent/avance pushing up, refund/cheap pushing down |
Real CSVs have text columns (comments, descriptions) — the lab used to drop them on the floor |
| V25 — delivered | Scale: the lab now takes 100k–1M-row files without dying, on a measure-first design. Before: a stack overflow killed every run past ~65k rows (push(...spread) in the split), and the slow families made big runs unusable anyway (random forest alone: 535 s at 5 000 rows). Measured first, then fixed: (1) the split rewritten with plain loops; (2) the planned typed-array pipeline rewrite was descoped on measurement — the pipeline was never the bottleneck (parse 2.6 s + profile 4.9 s + fit/transform 3.1 s at 1M rows); (3) announced seeded sampling, never silent — a global cap of 100 000 usable rows (seeded, stratified, summary.sampledFrom) plus measured per-family training caps (forest 1 000 · tree 2 000 · logistic/linear/MLP 20 000 · k-NN 5 000 · GBDT 50 000) drawn as nested prefixes of one seeded order, every capped model scored on the same full test set, and every sample printed on the leaderboard, in the tuning panel and in the HTML report; k-NN's old silent internal 5 000-row subsample was folded into the announced mechanism; (4) a named memory guard: parsing streams and refuses past 20M cells (rows × columns) with the numbers spelled out, instead of letting the tab die. After, measured: a 1M-row file trains the whole 8-model zoo in ~130 s (and a 120k-row file in the same ~126 s — cost is flat past the cap), gbdt reaches 0.991 accuracy on the 50 000-row announced sample. Every demo dataset sits under every cap: existing behavior unchanged. 283 unit tests (announced-sampling determinism, stratified nesting, cap recording), 54 e2e (a generated 120k-row CSV trains with the announcement asserted; a 21M-cell file is refused by name) |
|
| V26 — delivered | Learning curves: the lab answers the classic budget question — "would more data help this model, or is it time to work on features?" — with one new chart. On demand (like tuning), one model is retrained on growing seeded fractions of the train split: the SAME nested prefixes V25's announced caps draw from (same seed, same order), so for a capped family the last point is exactly the leaderboard model's diet and the verdict says out loud whether the announced cap costs accuracy. A geometric ladder of up to 6 sizes (each at least 16 rows, refused entirely when only one rung fits — one point is not a curve); at every size the pipeline is REFITTED on that prefix only (imputation, encoding, IDF, scaling all see just those rows — the strict no-leakage reading of a learning curve) and the model is scored on the same full held-out test set with a V20 bootstrap 95% band. The verdict is the V20 paired bootstrap applied to the last size step: a decisive gain reads "still climbing — more data would probably help", anything else "flattened — work on features or the model", each with the capped variant ("the cap costs accuracy" / "the cap costs nothing here"). The chart (log-spaced sizes, CI band, one dot per announced size) ships with its numbers table, joins the run artifacts (history, HTML report, share links) and refuses the baseline by name — flat by definition. Measured on Titanic/gbdt: 45 → 713 rows traces 0.753 → 0.820 with the plateau verdict at the last step. 293 unit tests, 55 e2e | |
| V27 — delivered | Local chat, upgraded: a real language model — Qwen3-0.6B-DQ, 355 MB, Apache-2.0 — running entirely in the browser, offered beside the V6 deterministic interpreter, which stays the DEFAULT and the fallback. The model never computes: it translates a question into a V6 query, and every number still comes from the deterministic engine. Its output is checked against the closed Intent grammar — invented columns, unknown operators, absurd k, a correlation of a column with itself are all REFUSED, and a refusal falls back to the keyword parser, with a badge under each answer naming which engine produced the query. Three constraints shaped the build, all discovered by measurement: (1) Cloudflare Pages refuses assets over 25 MiB, so scripts/prepare-llm.mjs fetches the weights at DEPLOY time (never committed — 355 MB in git would slow every clone) and splits them into 15 parts of ≤ 24 MiB, glued back in the browser through transformers.js's customCache hook, with per-part size checks and named refusals (llm-part-missing, llm-part-size, llm-short); (2) the strict CSP forbids the library's CDN default, so its pinned ONNX Runtime build is self-hosted under /ort-llm/ — the jsep (WebGPU) variant clears the 25 MiB limit by only 0.1 MiB, so re-check it on upgrades; (3) WebGPU is required, not preferred: the model's GatherBlockQuantized embedding kernel has no WASM implementation and needs shader-f16, so a device without it gets a named refusal instead of an unusable download. Verified end to end in a browser: the sharded weights download with correct cumulative progress, reassemble, and build an ONNX Runtime WebGPU session (18 s warm). Honest limit, stated rather than hidden: the interpretation-quality bench could NOT be run here — it needs a GPU with shader-f16, which the CI and dev runners lack. It ships instead as a repo tool (npm run llm:bench, 16 FR/EN questions over Titanic, half of them phrasings the keyword grammar cannot catch) that exercises the real production path, so the number can be measured on real hardware before the model is promoted beyond opt-in. 314 unit tests, 57 e2e |
|
| V27.1 — delivered | The model earns its place, it does not take it: the V27 order was wrong, and the measurement said so. With the local model selected it read EVERY question first and won whenever its JSON passed the grammar check — even when the keyword parser had a correct reading of its own. Measured in production on six reference questions over Titanic: the model turned « combien de personnes sont montées à Cherbourg ? » into embarked = Cherbourg → 0 rows, where the deterministic parser had embark_town = Cherbourg → 168; and read « est-ce que les femmes payaient plus cher que les hommes ? » as a plain count (314 female) instead of mean fare grouped by sex. Tally: 2 right, 2 confidently wrong, 2 refusals. The order is now deterministic first, model as a rescue (resolveIntent, unit-tested): the keyword grammar can only ever name a column that exists and a value that actually occurs in it, so when it understands, nothing overrides it — and the model is asked only about what it gives up on, which is exactly the gap that justifies its 355 MB. On the same six, measured on the owner's GPU after deploy: 5 right, 1 wrong, 0 refusals — up from 2 right, 2 wrong, 2 refusals. Two further defects fixed: (1) a refusal was badged « question read by the local model », claiming a reading nobody had made — refusals now name nobody and say whether the model was even consulted; (2) the system prompt had no groupBy and no top-k example at all, and no rule tying a filter value to the column whose value list contains it — both added, with FR phrasings and a numeric-threshold example. The bench gains the two shapes that failed (age < 10, a top-k) and now reports the shipped order as its headline number instead of the two engines separately. 326 unit tests, 57 e2e. The number that justifies the download: « combien d'enfants de moins de 10 ans ? » → count age < 10 = 62 and « à quel âge moyen voyageaient les passagers ? » → mean age = 29.699, both of which the keyword grammar refuses outright; and « combien de personnes sont montées à Cherbourg ? » came back as the deterministic engine's 168, the model never consulted. Still open: one question of the six — « est-ce que les femmes payaient plus cher que les hommes ? » — is still read as a correlation (fare↔age, a column the question never names); addressed in V27.2. The full bench remains un-runnable here (no shader-f16), so its number still has to come from real hardware. |
Measured by the owner in production (22/08/2026), the day V27 shipped. A confidently wrong answer costs more trust than a refusal — and V27 produced two of them, including a 0 where the deterministic engine already had the right 168. |
| V27.2 — delivered | Two honesty defects, one measured, one found while reading the measurement: (1) the comparison question V27.1 left wrong — « est-ce que les femmes payaient plus cher que les hommes ? » read as a correlation between fare and age — gets a rule that names both halves of the mistake: a question comparing two groups is an aggregate with groupBy on the column whose values name them, NEVER a correlation; and never pick a column the question does not mention. A second FR comparison example ships with it, in a different phrasing from the failing one, which stays a held-out bench case rather than becoming a memorised answer. (2) The answer sentence said « (sur 891 lignes) » under a mean built from 714 values: rowsConsidered counts rows after the filter, while numericAt skips missing and unparseable cells. The number was right, the sentence around it was not. Aggregates now carry valuesUsed (scalar) and used per group, set only when they differ from the row count, and the UI says « 714 valeurs utilisables sur 891 lignes » — matching what the correlation branch already did. This one predates V27 entirely: it has been there since V6. 330 unit tests, 57 e2e. Measured after deploy: the rule did stop the correlation — but the model then read the same question as count fare >= 0, still wrong. Two prompt attempts, two failure modes; see V27.3 for where that stops. |
Measured by the owner on real hardware (22/08/2026): 5 of 6 reference questions right after V27.1. The sixth is a confidently wrong answer to a different question than the one asked, and the « sur 891 lignes » wording was found by reading that same screenshot closely — a right number inside a wrong sentence is exactly what this project refuses to ship. |
| V27.3 — delivered | A >= that was quietly an =: retesting the comparison question after V27.2 produced « 0 ligne correspond où fare >= 0 » — impossible on a table where all 891 fares clear zero. Root cause found by reproduction, not by reading: the model emitted "value": "0" as a string, asFilter accepted a string for any operator, and matchesFilter took the numeric branch only for typeof value === 'number' — so >= fell through to the equality branch and tested fare == "0" against a column whose zero fares are written 0.0. Same intent with a real number: 891 rows. The hole is closed on both sides: asFilter converts a numeric string and refuses anything else on <, <=, >, >= (equality keeps text — that is how categorical filters work), and matchesFilter handles the ordering operators apart, throwing the named filter-not-numeric rather than passing a bug off as a query with no matches. V6 code, reachable only through the model: the keyword parser always built numbers. And a limit, recorded rather than papered over: « est-ce que les femmes payaient plus cher que les hommes ? » is still read wrong — a correlation before V27.2, a vacuous count after. The model finds fare every time and the shape never. Two prompt attempts are enough; a third would be sewing the prompt around one sentence, which buys a flattering bench and nothing else. The measured score stands at 5 of 6, and the sixth is written down as what a 0.6B does not do. 337 unit tests, 57 e2e. |
Found by retesting in production (22/08/2026). An arithmetically impossible answer — zero rows for a condition every row satisfies — is worse than a refusal and worse than a wrong reading: it makes the engine itself untrustworthy, which is the one thing LabML sells. |
| V28 — delivered | « Ne nous croyez pas sur parole » — a /privacy route that states the local-only promise once, in full, and then hands the reader the means to check it without trusting a word of it. Four verification steps, ordered by how hard they are to fake: cut the network (DevTools → Network → Offline, or the Wi-Fi switch) and watch the whole lab keep working; watch the Network tab while loading a file and training, and see nothing happen; read Content-Security-Policy on the document itself; open Application → IndexedDB and see exactly what was kept. The served policy is quoted verbatim on the page and pinned to public/_headers by a unit test — a page that claims a protection the site quietly dropped is worse than no page. A schematic of the Network panel is drawn rather than screenshotted (DevTools chrome differs per browser and per locale) and captioned as a diagram, not a capture. A live audit panel counts this page's own resource timings by origin and says, in the same breath, what it cannot see: worker timelines and requests the CSP blocked — a proof that oversells itself is worth less than none. Last section lists what does cross the network (app files, demo datasets on click, vision models on entering Vision, LLM weights on explicit consent) and what never does. FR/EN, prerendered shell, WCAG AA verified by axe including the audit result. 344 unit tests, 60 e2e. |
Owner request (22/08/2026): the promise is repeated across the site, but a user has no way to tell a true claim from a comforting one. Verifiability is the product here — anyone can write « your data stays local » in a footer. |
| V29 — delivered | Analytical SQL in the browser (DuckDB-Wasm, MIT): the Data Studio gains a real OLAP engine — joins, window functions, aggregations — over the file you just loaded, with no server and no upload. The file is queried as dropped, before the cleaning recipe: the recipe belongs to the studio, and a result traceable to nothing the user can reopen would be worse than no SQL at all. Extra CSV / Parquet / JSON files can be attached in the same session (Parquet is a new input format for the lab), each exposed as a view named after the file; a result exports to CSV or goes to the ML Lab in one click, through the handoff path V4 already built. Errors show DuckDB's own message — it names the line and the token, which no paraphrase of ours would. The measurement that set the version: @duckdb/duckdb-wasm is pinned to 1.28.0, not latest. From 1.29 the binaries cross Cloudflare Pages' hard 25 MiB per-file limit (eh 34.2 MiB, mvp 39.4 MiB); at 1.28.0 they are 17.3 and 21.1 MiB and fit. Newer would have meant sharding the wasm and either widening connect-src to blob: — days after publishing a page that quotes that very directive — or rebuilding the service worker in injectManifest mode. An older engine was the cheaper honest trade, and it is written here so the next upgrade re-measures instead of rediscovering. Self-hosted under /duckdb/ (the library defaults to jsDelivr, which the CSP refuses), never precached — cached on first use like the vision models, so nobody pays 18 MiB before opening the console — and the coi threaded build is left out entirely: no COOP/COEP, no SharedArrayBuffer, single-threaded as the assumed mode. Remote S3/HTTP querying stays out, by CSP and by intent. 352 unit tests, 61 e2e. |
Owner request (21/08/2026): real analytical SQL on ~100 MB files with zero backend. Delivered after the /privacy page at the owner's request (22/08/2026). |
| V30 — delivered | Chat that reads better, measured before it is made bigger. The wave began by building the instrument, because V27's stood on 18 cases that needed a GPU with shader-f16 — one laptop's worth of evidence, re-runnable by nobody. It now stands on 55 reference questions, French and English, over every shape of the query grammar plus three that no query can answer, where refusing is the only correct outcome. Two harnesses run it: one in CI on every commit with no model at all (the deterministic parser, the grammar automaton, the token mask), and one against the REAL pinned q4f16 weights on a CPU through onnxruntime-node — same files, same prompt, same decoding path as production, minus the GPU. « Measurable » stopped meaning « on one machine ». The instrument immediately contradicted the wave's own premise. The failure was not mainly the model: on those 55 questions the shipped app answered 33 right, 15 WRONG, 7 refused — and seven of the fifteen wrong came from the deterministic parser, which runs first and can never be overridden. « Combien de femmes ? » answered 891 instead of 314: the grammar knows combien, knows nothing about femmes, kept the count and dropped the condition — under the badge that is supposed to mean exact. Four of those seven the local model reads correctly, and never got asked. So the parser now checks its own coverage: every word of the question must be accounted for by a lexicon phrase, a column the answer uses, a value it filters on, or one of three closed lists (the table's own furniture, generic row nouns, grammatical filler). A leftover word is a refusal. Measured: 19 right, 0 wrong, 36 refused — the seven wrong answers became refusals and not one correct answer was lost. wrong === 0 is now asserted in CI, and the trade is one-directional by construction: an unknown word can cost a refusal where an answer was possible, never a wrong answer where a refusal was right. (B) Constrained decoding, hand-written: an automaton over the query grammar and a LogitsProcessor that masks, at every token, everything that would leave it. It walks UTF-8 bytes, not characters, because Qwen's vocabulary is byte-level BPE and 1 457 of its 151 669 tokens are fragments of a character — a character-level automaton would have made « Île-de-France » unwritable as a filter value. It cost about 16 ms at its most expensive step ({"kind":" masks seven letters against seven large buckets) after the first-byte step was hoisted out of the per-token loop, down from 53 ms. And on its own it made things worse: model refusals fell 14 → 2 and correct answers rose 29 → 34, but wrong answers rose 12 → 19. Forcing a valid answer turns « I could not parse that » into a confident wrong number. That is why the grammar keeps {"kind":"none"} reachable — a shape whose only meaning is « I cannot express this », which maps to the refusal V27 already had. (C) Examples drawn from the user's own columns, for 0 MB — and the reason turned out to be sharper than the plan's. V27's nine examples were frozen Titanic, and seven of the 55 corpus questions appear in them verbatim: the prompt had been fitted to the bench across V27.1 and V27.2, so on those questions the old bench could not tell reading from recitation. (Checked rather than assumed: on those seven, before and after score identically, 5 right / 1 wrong / 1 refused. The defect is methodological, and its measured effect on this comparison is zero.) Generating the examples from the loaded file removes the contamination structurally and deletes the rule that asked the model to ignore what it had just been shown. Two versions of them were worse than the frozen ones, and both reasons are now in the code. The first left out the aggregate-WITH-FILTER shape: under constraint, a shape the model has not been shown comes out as a confident wrong answer rather than a refusal, and « prix moyen payé par les survivants » became count where fare = 1000000000. The second still picked the FIRST numeric column, which on Titanic is survived — so the examples read « average survived » and the model duly reached for that column on questions that never mention it. ColumnInfo gained a capped distinct count so an example averages a quantity, not a 0/1 flag. (D) Two samples, one vote — dropped, on this wave's own measurements. Both halves of its tie-break died with (B): constrained decoding guarantees every candidate validates, so « keep the one that validates » no longer discriminates; and « invents no column the question never names » is contradicted by the corpus, where « did women pay more than men? » is correctly answered with fare, a column the question never names. Two samples for a vote with no criterion, at twice the latency, is not a trade. What the wave deliberately does not do: ship a second, bigger model as a download (the cheap levers were not exhausted when the plan proposed it, and now they are), guess a column by fuzzy name-matching (the refusal is the honest outcome), or let the grammar automaton replace validateIntent — it over-approximates in two named places and is a filter, not the authority. 557 unit tests, 79 e2e. |
Owner question (22/08/2026): would a bigger model raise the share of correct answers? The wave answers with a measurement rather than an estimate, and the answer has two halves. For 0 MB, the app went from 33 right / 15 wrong to 42 right / 7 wrong out of 55 — nine more correct answers and fifty-three percent fewer wrong ones, the single largest piece of which came from the deterministic parser rather than the model. And the second half was measured too, not deferred: Qwen3-1.7B at 1.43 GB — four times the download — scores worse (40 right / 12 wrong against 42 / 7). It reads the hard questions better and the easy ones worse. « Bigger » is not a direction of improvement on this task; it is a trade whose sign has to be measured, and the bench now measures it in one command. |
| V31 — delivered | Vision that says « I do not know » — and a bench that refuted three of this row's own predictions. (A) Measure first, as V30 taught: the complaint « it still makes mistakes » is not a measurable statement. The bench is 14 images, not the 30–50 the plan asked for, and the shortfall is deliberate: a search-and-download pipeline produced a backlit dog, a triptych of broccoli close-ups and an eighteenth-century painting of a lighthouse, each of which would have scored the model wrong for the corpus's own mistakes. Every kept image was looked at, and the ones whose ground truth could not be stated were thrown away — fewer and verified is an instrument, more and unverified is noise with a percentage attached. Licences are CC0, public domain or CC BY only, never CC BY-SA, whose share-alike term would propagate into an MIT repository. It runs in the ordinary e2e suite, in the real browser, through the real canvas and the real worker, in about 15 seconds. What it found: where an ImageNet label exists at all the classifier is 9/10 top-1; the object detector is 7/9; face counts are 10/10 exact. And on the four images no label in the thousand can name, the page answered anyway — « stage » at 36.7 %, « golf cart » at 37.1 %, « semi-trailer truck » at 41.2 %, « football helmet » at 86.6 %. (C) The honest refusal, for 0 MB. A softmax over 1000 classes cannot abstain — it renormalises to 1 whatever it is shown — so the refusal is decided outside the classifier, by two rules that were each chosen against the measurement rather than picked and hoped for. Rule 1, the human subject: fire when the object detector finds a person AND the face detector finds a face. Requiring both is what the bench bought — YOLOX drew a person box on a wine bottle and on a red sports car, and UltraFace found no face on either, so a person box alone would have cost two correct answers. Rule 2, a confidence floor at 50 %: on this corpus the cheapest correct answer is 70.5 % and the dearest wrong one not already taken by rule 1 is 41.2 %. Rule 1 wins over rule 2, because « football helmet » at 86.6 % walks straight through any floor. Measured: 4/4 unnameable images refused, 1/1 wrong answer announced as wrong, 0/9 correct answers lost — both guarantees frozen as assertions in the bench. The label is never hidden, only framed: the top-5 stays on screen with its probabilities, because a number nobody can see is a number nobody can check. Three of this row's own predictions were wrong, and the measurement said so. (i) « Squashing a 16:9 photo into a square skews everything — preprocess.ts is the suspect »: the code never squashed, it centre-crops, and replacing the crop with a full-frame squash scores identically, 9/10, with the same single wrong answer. The crop is not the defect. (ii) Averaging two crops therefore has nothing to buy — both views agree on the failure. (iii) Recalibrating OBJECT_THRESHOLD: probed at 0.05, YOLOX-Nano does not see the football at any threshold (its best guesses on that image are « toilet » 14 % and « toaster » 8 %) and does not see the bottle either. Lowering the bar recovers neither miss and admits noise; the V23 value stands. The squash experiment also produced the strongest argument for rule 1 existing at all: under it, the photograph of people under an umbrella comes back as « umbrella » at 99.3 % — plausible, confident, and not what the picture is of. No confidence floor would ever catch that one. (B) CLIP ViT-B/32 (~190 MB) — not in this wave, and for a reason the bench can state precisely: the defect this wave targeted is fully covered for 0 MB, and CLIP would answer a different question. The refusal turns a false answer into an honest silence; it does not turn it into a right answer. Whether 190 MB is worth turning that silence into a name is a wave-scale decision, and the bench cannot arbitrate it, because it scores a task CLIP would replace rather than improve. (D) YOLOX-S (~35 MB) — not in this wave either, and this one the bench does arbitrate: the two object misses are inert. Both images are already named correctly by the classifier, and both of the detector's false person boxes are already neutralised by rule 1's face requirement, at no cost. That is +31 MB, plus the < 25 MiB splitting machinery V27 needed for Cloudflare Pages, to buy two chips on two images and nothing at all for the verdict. What this wave deliberately does not do: use the detector to correct the classifier (« the detector says cat, so the label French Bulldog is wrong » is a COCO-to-ImageNet taxonomy problem, and that image is already caught by the floor — a third rule with zero measured effect is an untested parameter, which is exactly what V30 warned against), hide the label behind the verdict, or present the 50 % floor as a law: it is estimated from fourteen images, the gap it sits in is wide but weakly sampled, and it is written down as a named constant so the next corpus can move it. 566 unit tests, 80 e2e. |
Owner report (22/08/2026): the vision playground is better than the chat but still makes mistakes. Naming the label-space mismatch is what turns a vague complaint into a fixable defect. |
| V32 — delivered | Documentation that cannot lie — and a measurement that rewrote this row's own rule. A /docs route, linked from the footer, built on the Diátaxis split, with the Markdown living in src/content/docs/<lang>/*.md and compiled at build time: the reader downloads finished pages, an outline and a search index — never a parser, and never a request to a documentation host. Search is local for the same reason the models are self-hosted: a site whose whole claim is that nothing leaves cannot make its own docs the exception. At this corpus size the index is a scan, stated as such rather than dressed up. The wave began by measuring, and the measurement is what made rule (1) buildable. « The docs are tested like the code » only works if you know which numbers are stable, so the tutorial's exact path was run twice before a word was written: every metric came back identical, and every wall clock did not — Random forest trained in 12 050 ms and 13 231 ms on the same machine, minutes apart. A page quoting « trained in 0.7 ms » would therefore break the build with nothing broken. That is now its own e2e test: no documentation page may quote a duration, in any language, ever. The measurement also corrected this row: the plan illustrated « you will get 0.821 accuracy » and the real champion figure is 0.792. The tutorial teaches the thing the app already does honestly: on titanic the elected champion is k-nearest neighbors at 0.818 on validation, and it is only sixth on test at 0.792 — the decision tree scores 0.831. A tutorial that printed 0.831 in large type would be lying about its own method, so the page makes the −0.026 selection gap its central lesson instead of a footnote. The guard was wrong twice before it was right, and both failures were found by attacking it. Version one asserted the page CONTAINS the right figures — so editing one occurrence of 0.792 into 0.800 still passed, because the other occurrence kept the assertion true. Version two read the figures OUT of the page and demanded the app produce each one, but scoped that to blockquotes and tables — so the same edit in bold prose still passed, and bold prose is where the reader actually takes the number from. Version three covers quotes, tables and bold, and was re-attacked on all three: a wrong figure in English prose, a wrong figure in a French table, and a quoted duration each fail now. Its limit is stated in the spec rather than left to be discovered: plain prose is not checked, because prose also carries numbers that are reasoning rather than quotation (« ten minutes », « 62% of the rows »), so the convention is that a figure you want checked goes in a quote, a table, or bold. (3) Better than a screenshot, a link that does the thing: :::try /ml?demo=titanic | label compiles to a deep link, and /ml now reads ?demo= and ?target=. A screenshot is a claim about the past that rots silently; a deep link either works or the e2e catches it. The parameter becomes a fetched path, so it is resolved against the shipped demo list and nothing else — an e2e test asserts that ?demo=../../../etc/passwdfetches nothing. What this wave deliberately does not do: write the reference pages before the template is settled (rewriting all of them is the predictable cost), hand-take a screenshot, or claim the figure guard is total.marked is a build-time devDependency and never reaches the browser. 601 unit tests, 86 e2e. |
Owner request (22/08/2026): document every shipped feature across /ml, /data and /ai, linked from the footer. One finished tutorial first, on purpose — writing the full reference before the template is settled means rewriting all of it. |
| V33 — delivered | The reference, and a table of refusals extracted from the code rather than from memory. Five pages per language — the refusals table, ML Lab, Data Studio, Vision & assistant, and file formats — grouped by section rather than one page per panel, so a lookup lands on one page with anchors instead of hunting across twenty. The wave began by inventorying, not by writing. A refusal table written from memory documents the memory; this one was extracted from the source: 38 kebab codes are thrown, 2 of them only by tests (constrained-decoding-unavailable in the LLM bench, header-not-served in the privacy test) and are therefore absent from the public table, because a row nobody can encounter is padding. The four codes this row names all exist. refusals.ts lists every shipped code with its audience, and refusals.test.ts re-extracts them on every run in both directions: a code thrown but unlisted fails, a code listed but no longer thrown fails. Both were proved by breaking them on purpose. Two further tests tie the catalogue to the pages — every visitor-facing code must appear in both languages, and no page may name a code the code does not have. The inventory found a real defect, which is what an inventory is for. dataset-missing — a kept dataset that IndexedDB no longer holds — had no message of its own and fell through to « the file looks empty or not tabular. Try a CSV with a header row », sending the reader to hunt a format problem that does not exist. Worse than an undecodable refusal: a mis-decoded one. It now says what actually happened. It also cleared two false suspects: not-parallelisable and not-serialisable are not refusals at all — parallel-run.ts drops them and the main worker trains that family sequentially, same result, only slower — so they are documented as guards that are deliberately silent rather than listed as failures. V32's figure guard had to be narrowed, and the measurement is why. It asserted that every figure quoted in any doc page appears in a live titanic run. That is right for a tutorial, which reproduces a run, and meaningless for a reference page quoting a 0.35 threshold or a 13.6 MB model. Run against the new pages it went green while genuinely checking nothing — it passed on coincidental small integers. It is now scoped by the front matter's Diátaxis kind, and the reference pages get their own guard: named constants are checked against the file that defines them, and the three vision model sizes against the bytes on disk. That guard caught a wrong figure in its own first run — the reference page said the three models weigh 18.6 MB, copied from this very row; they weigh 18.5. Its bound is stated in the spec: it checks the constants listed, not every number on a page. What this wave deliberately does not do: one page per panel (twenty pages make lookup worse, not better), generate the reference from the code (it produces mush, as this row predicted), or claim the guards are total. 611 unit tests, 88 e2e. |
A feature nobody can look up is a feature that does not exist for the reader; and a refusal nobody can decode reads as a bug rather than as the design it is. |
| V34 — delivered | The explanations, the how-to guides, and a limits page extracted from this very file. Six new pages per language — two explanations (the method choices; what LabML does not do) and four task-shaped how-to guides (score a batch, compare two runs, read a learning curve, hand a SQL result to the lab) — bringing the documentation to twelve pages per language across all four Diátaxis quadrants. The limits page is extracted, not remembered, the way V33's refusals were: PLAN.md records what every wave deliberately refused, and the page reads it back. That matters because memory flatters — elegant renunciations are easy to recall and embarrassing ones are not. The page therefore separates three kinds of limit that are usually conflated: a design choice (feasible, but it would have cost something better), a measured drop (built or costed, and the measurement said no), and a refuted prediction (the plan asserted, the measurement disagreed, and measurement won). The third section is the uncomfortable one and the most useful: it lists this file's own errors — the crop that was never a squash, « you will get 0.821 » against a real 0.792, 18.6 MB against a real 18.5, V27's inverted interpreter order. A guard traces each quoted figure back to the PLAN entry that recorded it, so the page cannot keep asserting a renunciation that was later reversed — which has already happened once: class weighting was descoped by name in V16 and delivered in V36, and the page says so, because that is what separates « not done » from « forgotten ». « No next step » is now a test, not a habit. Every page must end with « Et ensuite ? » / « Where to go next » AND offer at least one working link; the guard found ten of the twelve existing pages had no such section and the English tutorial had one with no links at all. Three more page-level guards came with it: every slug exists in both languages (a slug in one language only sends a language-switching reader to a « page does not exist »), every /docs/… link resolves to a real slug, and all four quadrants are represented. The guard's own bug was caught by the guard. It demanded the figure « 1.43 » in both languages and failed on the French page, which correctly writes « 1,43 » — a defect in the check, not in the page; it now accepts either decimal separator, and only that. What this wave deliberately does not do: repeat /privacy in prose (two texts on one subject diverge, and it is the forgotten one that survives — the method page links instead), ship hand-taken screenshots, or extrapolate a learning curve beyond what was measured. 672 unit tests, 88 e2e. |
Three audiences, deliberately: the curious visitor (five minutes), the practitioner (one task), and the evaluator judging whether the engineering is rigorous. The explanation pages are what the third one reads. |
| V35 — delivered | ML Lab: the number stops flattering itself. Two method defects in shipped code, fixed, plus the two additions that follow from them. (1) The winner was picked on the test set — the leaderboard sorted nine models by the metric computed on test and crowned sorted[0], which makes the headline figure the optimistic maximum of nine draws. There is now a third split: validation is carved out of the train side (64/16/20 with the default ratios), ranking and crowning happen on validation, and the champion line spells out both numbers and the gap between them — « selected on validation at 0.974, scores 0.917 on the untouched test set ». The test indices are byte-identical to what the same config produced before V35, so every panel that reads the test set (segments, thresholds, uncertainty, batch compare) is unchanged; below 60 usable rows the third split is refused by name and the lab ranks on test as before. Ranking now lives in ONE module (ranking.ts) used by the leaderboard, the history, the run comparison, the report and the auto-selected insights model — the bug that shipped mid-wave was exactly that a fourth site still sorted on test and opened a different model than the one crowned. (2) The split was always random, even on dated data. A chronological split (oldest rows train, newest test, rows without a parseable date dropped and counted) and a group split (no group on both sides — the same customer in train and test is the same leak) are offered when a column supports one, and announced in the run info. (3) A predictive leak detector: V6 caught columns that MAP to the target; this catches the merely predictive one — a one-column stump fitted on train and scored on validation, and a lone column reading the target at ≥ 99% shows as a copper warning with its measured score, never as a victory. (4) A robust leaderboard on demand: 5×2 repeated cross-validation over train+validation (the pipeline refitted inside every fold, the test set never touched), reporting a mean, a spread, and how often the leader actually beat the runner-up — « 10 of 10 folds: the order is stable » or « 6 of 10: treat them as tied ». A defect the wave exposed and named: with the smaller train split, Gaussian Naive Bayes on ~150 TF-IDF features saturates to exactly 0/1, so V24's word-effect occlusion measured exactly zero for every word and the card simply vanished — reading as « no word matters », which is false. Measured (2 distinct probabilities out of 48 test rows, against 48 for logistic and gbdt), the card now refuses by name and points at a model that can answer. 369 unit tests, 65 e2e. |
Owner request (22/08/2026), launched 23/08/2026. Two of the four items were defects rather than gaps: a lab that sells honest evaluation cannot ship a headline figure it knows to be optimistic, nor a split that leaks on dated data. |
| V36 — delivered | ML Lab: the gaps that were deliberately left open. Each item was consciously deferred in an earlier wave rather than forgotten; delivering them together keeps the descopes visible instead of letting them quietly become permanent. (1) Class imbalance, descoped by name in V16. Two mechanisms, each NAMED per family rather than hidden behind one word: the loss is weighted where the loss is ours (logistic regression, gradient boosting — the gradient AND the hessian are scaled, since scaling only the gradient inflates leaf values instead of rebalancing), and a seeded balanced resample is used for the ml-cart families (tree, forest), which take no sample weights. The minority is upsampled to the majority's size — never the reverse: balancing by trimming the common class throws away real observations to fix a ratio. Off by default, because on a balanced target it changes nothing and a knob that does nothing is worse than no knob; the run announces the majority share, and the leaderboard says so when it crosses 60%. (2) The ranking metric is a choice. Accuracy and RMSE were imposed, which is the wrong criterion on an imbalanced problem — a model that never predicts the rare class can top an accuracy ranking and be useless. Rank on F1, recall, precision or ROC-AUC and the order genuinely changes; a model that cannot produce the chosen metric sorts last rather than being dropped. Ranking stays in the single V35 module, so the leaderboard, the history, the comparison, the report and the auto-inspected model all move together. (3) Multiclass thresholds, open since V16, read one-vs-rest: pick a class, score it against all the others, same PR and calibration curves. What the panel refuses to imply is a complete multiclass decision rule — two classes can both clear their thresholds and nothing here says which wins, so it says that instead. (4) An ensemble of the best: the average of the top three, built from models already fitted, so it costs one pass over the test set. The baseline is never a member (averaging a constant predictor drags the result toward the majority class), members are picked by the same V35 ranking rule, and probabilistic members are preferred — a mid-wave measurement showed the ensemble winning on iris with a bare vote because k-NN was in the top three, which silently closed the threshold, calibration and word-effect panels on the champion. What this wave deliberately does not do: add a tenth model family (nine is plenty; a tenth improves neither honesty nor understanding), build an AutoML « we handle everything » mode (the opposite of a lab that shows its decisions), or bring in tabular deep learning (high cost, no gain at this scale, and no longer hand-written). 387 unit tests, 69 e2e. | Launched 23/08/2026, right after V35. Each item was a named descope, not an oversight — and the ensemble exposed one more silent-disappearance defect, of the same family as the one V35 found. |
| V37 — delivered | ML Lab: speed and the comfort of long sessions. The wave opened, as the plan demanded, with a measurement — and the measurement moved the wave. (1) Parallel training was the headline: the zoo trains sequentially in one worker, so the heavy families were shipped to helper cores. Models cannot cross a worker boundary — predict is a closure and structured clone drops functions — so each helper returns toJSON() as a JSON string and the caller rebuilds through the V22 import path, which makes a parallel model byte-identical to an imported one. That string is not a detail: posting the object instead let structured clone keep shapes JSON drops, and ml-cart's load() then rebuilt a tree whose first prediction threw this.root.classify(...).maxRowIndex is not a function — a defect that would only have surfaced later, when the user opened insights. Helpers are split by measured cost, heaviest first, to the lightest helper (greedy longest-processing-time); k-NN never leaves the main worker, being the one family with no toJSON and also the one that fits in 0 ms; any failure — no Worker support, a helper that throws, an unserialisable family — silently falls back to the sequential trainer, because parallelism may change how long a run takes and never which models it produces. Every family's inference latency is still measured here, on the rebuilt predictor: a helper's timing would describe another core under contention, and the column would otherwise read 0 ms for exactly the families that ran in parallel. (2) The measurement then found the real bottleneck, which was not training at all. On a 60 000-row run, k-NN inference cost 59.6 s of a 68.8 s wall time — 87% of the whole run — because the neighbour search allocated 5 000 objects and sorted them for every single prediction, and the scorer asked for labels and then for probabilities, searching twice. Fixed with a bounded top-k insertion over a flat Float64Array (ties keep the row seen first, exactly as the stable sort did) and an explicit predictWithProba for families where both answers come out of one computation. The old sorted implementation is kept verbatim in the tests as the oracle: the fast path is asserted to predict identically, row for row. (3) Comparing more than two runs — V21 compares two; three to six read against the oldest of the selection, so the deltas say what the session's changes did rather than what the newest run happens to be. Deliberately not a second diff engine: the matrix is the same V35 ranking and the feature columns are set algebra over the same summary.featureColumns V21 reads. Measured, same machine, same 60 000-row file, same seed — four arms, reported in « V37 — parallel training and the k-NN bottleneck » below this table. So parallel training alone was worth 1.06× — real work, drowned by a bottleneck nobody had measured; the k-NN fix alone 5.0×; parallelism on top of it 1.54×; together 7.8×, with every leaderboard number identical. No minimum dataset size guards the helpers, because that was measured too: the gain is already positive at 1 000 rows (2 919 → 2 375 ms), at 3 000 (6 479 → 5 446 ms) and at 8 000 (7 496 → 5 937 ms). Descoped by name: resuming an interrupted run. It was the third item and it is not shipping, for a stated reason rather than for lack of time. A resume needs the fitted pipeline persisted as well as the models; k-NN has no toJSON at all, so the resumed leaderboard would be missing a family the interrupted one had; and it would only work for runs whose dataset the user opted to keep under V19's 50 MB budget — a feature that works for some runs and silently produces a different leaderboard for others is worse than no feature. The exposure it protects against also shrank by 7.8× in this very wave. It stays open, honestly, rather than shipping as a half-feature. What this wave deliberately does not do: cache trained models across runs (the cache key is the entire config plus the data — a stale hit is a silently wrong leaderboard), or move k-NN scoring to a helper (it is now 700 ms, and the model must exist in the main worker anyway for insights and what-if). 422 unit tests, 71 e2e. |
Launched 23/08/2026, after V36. The plan said « measure before and after so the gain is published rather than claimed » — and the measurement is what turned the wave around: the promised parallelism was worth 6%, while the bottleneck it revealed was worth 5×. Two latent defects fell out of the same instrumentation: models corrupted by structured clone, and an inference column reading 0 ms for every parallel family. |
| V38 — delivered | Data Studio: reading the file exactly as it was written. The headline item was a defect in shipped code, not a missing feature, and the wave opened by proving it. Same 900 rows, same three numeric columns, same seed — only the way a number is spelled changes — the figures are in « V38 — the same file, two spellings » below this table. The baseline scores 0.594, so the headroom above it collapses from 0.406 to 0.225: 45% of the achievable gain, lost in silence. parseNumber ends in Number(cleaned), Number('12,5') is NaN, the column falls through, and V24's TF-IDF cheerfully tokenises digits into a hundred word features. Nothing warned, nothing refused — the pipeline just produced a worse model. Three of this row's own predictions were wrong, and the measurement corrected them: the mis-typed column becomes text on high cardinality but categorical (one-hot) on low, not text alone; a windows-1252 file does not display Québec — that is the reverse case (UTF-8 read as cp1252) — it yields U+FFFD replacement characters; and day-first dates were already handled by V8, 31/12/2025 parsing correctly all along, with only the dash form 31-12-2025 returning null. The fix. One reader, shared by the ML Lab and the Data Studio, that decides by evidence rather than by guessing the user's locale: the browser's language says nothing about the file someone dragged in. Encoding is settled by trying and failing — UTF-8 in fatal mode throws on cp1252 accents, so the fallback is a certainty, not a preference; the delimiter is the candidate that splits every sampled line into the same number of columns, quotes respected; and the decimal separator is decided per column, never per file. A column is rewritten only when at least 90% of its values are numbers in that form AND it carries a comma AND it would otherwise not be numeric at all — so a text column containing « vis, tête plate » comes out untouched, and a column of bare integers is left alone because rewriting it would be a change with no cause. Values that do not match the pattern are never rewritten: a stray « n/d » stays « n/d » rather than becoming a plausible-looking number. Detection reads only the head of the file, so V25's streaming abort survives intact and a 2 GB file still stops at the cell budget. The reading is then announced with its evidence — « virgule décimale détectée et convertie dans 2 colonnes : surface (400/400), prix (400/400) » — above a five-row preview, and only when the reading was not the plain default: an ordinary UTF-8 comma file gets no card, no confirmation step, no friction. Batch scoring, drift and join files go through the same reader, because comparing a French export against a normalised dataset would otherwise report drift that is nothing but a decimal separator. What this wave deliberately does not do: guess a locale from navigator.language (the file has no relationship to the browser's language), rewrite a column on a bare majority (below the 90% floor the evidence is not evidence), or touch values individually inside a column the evidence does not cover. 446 unit tests, 73 e2e — of the 21 new reader tests, five assert that it refuses to act. |
Owner request (22/08/2026): what to improve in /data. The audit found a defect first, and the wave began by reproducing it end to end: a French-locale CSV — the single most likely file this owner's users will open — silently loses every numeric column. The repair is exact rather than approximate: the French file now trains to the same types, the same feature count and the same score, to ten decimal places, as the file that never had the problem. |
| V39 — delivered | Data Studio: a recipe that works column by column. RecipeOptions applied missing and clipOutliers to the whole file — one strategy for every column, however different they are. A median makes sense for an age and none at all for a postcode. The recipe is now an ordered list of per-column steps, with the file-wide settings demoted to defaults a column may override: a column with no entry behaves exactly as it did before V39, which is what keeps every previously exported recipe valid and replayable. The missing strategies that were absent are there: median, mean, most-frequent, a constant used verbatim, and a « MANQUANT » category for categorical columns — absence becoming a level of its own rather than a guess. Clipping became a per-column decision on the same terms, so a column can opt in or out of the global flag. The rule the wave exists to enforce: imputing without marking destroys information. A blank field is rarely blank at random, and the fact of the blank is frequently predictive on its own, so every column may add a <column>_absent indicator — and the ordering is the whole point: every indicator is written before any blank is filled, so it records where the blanks really were, never where some other column's rule left them. Three consequences fall out of that and are tested: a column with nothing missing gets no indicator (an all-zero column is noise, not information); rows dropped by one column's rule are gathered and removed once, so indicators stay aligned instead of being shifted by a sibling column; and when a strategy cannot be honoured — a median over a column holding no parseable number, a constant with no value typed — the blanks are left blank rather than filled with something invented. Because the indicator is optional, the studio announces the columns it filled without marking, by name, instead of quietly producing a tidier table. What this wave deliberately does not do: make the indicator mandatory (it would silently add columns to every existing recipe, and imposing is not the same as announcing), impute from a model (opaque, and it fabricates values that look plausible — already refused for V40), or reorder the recipe's fixed stages, which is what keeps toggling an option from compounding with the last one. 461 unit tests, 75 e2e. |
A single global strategy is the kind of default that looks tidy and quietly makes the data worse; per-column steps cost little to build because the recipe was already an object, not a pile of checkboxes. The build confirmed it: the engine change is contained in one stage of applyRecipe, and the 17 pre-existing recipe tests passed untouched. |
| V40 — delivered | Data Studio: validity, drift, and an auditable diff. Quality was measured as completeness and consistency of type; what was missing is validity — a value can be present, correctly typed and still impossible. Five named rules now say so in plain language: an age outside 0–120, a date in the future, a percentage outside 0–100, a negative amount, a malformed postcode. Then cross-column consistency: an end date before its start, a total that is not quantity × price — every cell fine, the ROW impossible. The plan's choice of engine was wrong, and building it showed why. It proposed running the consistency rules through V29's DuckDB, since the file is already registered there. But DuckDB is an announced, opt-in 18–22 MB download: routing these checks through it would have made a universally applicable check conditional on a large download most users will decline, leaving the panel empty for them. Comparing two columns of a table already in memory is a loop, so it is a loop, and every user gets it. DuckDB keeps the job it is genuinely needed for — arbitrary SQL, and the new Parquet export (COPY … TO, one call, bytes straight to a download). Both families of rules obey the two laws V38's reader established: they fire on evidence, never on a column's name alone — a column called age holding 20 000 is a duration in days, so the rule checks that most of the column is plausible before it flags the rest, and stays silent otherwise — and they report without ever repairing, because V39's recipe is the one record of what was done to the data. Three things then make the studio auditable rather than merely helpful. A before/after diff naming which rows, which columns and which values changed: the hard part is that a recipe drops rows and adds columns, so applyRecipe now returns which SOURCE row each surviving row came from — without it the diff would pair row 7 with a different row 7 and report a screen full of changes that never happened. A replayable reference profile, the V22 manifest idea applied to data: bin edges and shares, never rows, so a profile of a payroll file describes the shape of the salary distribution and nobody's salary — which is what makes it safe to commit beside the code, and it scores a new file to the same PSI, to six decimal places, as V11's live two-file comparison. And a score broken into its parts, each with its weight and what it actually cost. The weights now sum to 105, not 100, deliberately: validity brought its own 5 points rather than taking them from an existing part, because redistributing would have quietly changed what every previously published score meant. What this wave deliberately does not do: a spreadsheet-style cell editor (hand edits break reproducibility — the recipe is the record), fuzzy deduplication (guaranteed false positives on names and addresses, silently merging two real people), or model-based imputation (opaque, and it fabricates values that look plausible). 502 unit tests, 78 e2e. |
Came last because it builds on V38's faithful read and V39's per-column recipe: validity rules on mis-parsed numbers would have flagged the parser, not the data. Closes the Data Studio group. Of the 36 new unit tests, eleven assert that a rule REFUSES to fire — a rule that flags a good file is worse than no rule, because it teaches the reader to ignore the panel. |
| V41 — delivered | The site says what it is. Six corrections to what the site declared about itself — to crawlers, to link previews, to visitors — none of them touching the lab, all measured on production first (29/09/2026, curl). (1) The twelve /docs/<slug> pages the sitemap advertised were served by the root fallback: the home page's title and Open Graph tags, a canonical pointing at /, an empty <div id="root">. To a crawler, twelve duplicates of the home page; to a reader pasting a tutorial link in a chat, a preview of the wrong page. V35's guard checked the section shells and the sitemap's status codes and missed it, because a fallback answers 200 too. Each documentation page is now prerendered as docs/<slug>.html — Pages serves that at the clean URL the sitemap and the app's links already use, where a directory would have added a redirect — carrying the compiled article, its title, its summary and its own canonical; shells.spec.ts now reads every URL the sitemap lists. The twelve shells stay out of the service worker's precache (~130 KB each of inlined stylesheet and fonts, for pages the app already renders offline from its bundled DOCS module); the extglob docs/!(index).html spares the section index, which the plain docs/*.html had dropped too — 96 → 95 precache entries, measured. (2) The home description opened mid-sentence — « entirely in your browser. Drop a dataset… » — because only the highlighted half of the title was prepended to the lede. (3) / was the one section painting nothing before JavaScript, and the one most visitors land on, because the root file was also the SPA fallback (V14: « the root stays the SPA fallback (accepted) »). That acceptance is reversed: the root carries the home hero with HomePage's exact classes, and the fallback role moves to two new files. (4) A real 404. wrangler pages dev reports the file's single rule, /* /index.html 200, as invalid and ignored — Pages strips /index.html from URLs, so the rewrite would loop — and the 200 every unknown address answered came from the platform's default single-page fallback: every misspelled URL was a soft 404, and so was every missing asset, answered with HTML. A 404.html (the not-found hero, noindex) turns that fallback off; _redirects lists only the four routes with no file of their own — a run, a comparison, a share link — rewritten to the bare shell.html at its clean URL /shell (/shell.html itself answers a 308), exact source first, splats after, and the service worker's navigation fallback moves to the same shell so an offline run page no longer flashes the home hero. A unit test derives that list from router.tsx and the shell routes, so a route added on one side fails on the other; and a new Playwright project runs routing.spec.ts against wrangler pages dev, the one server that applies _redirects, _headers and 404.html the way production does — vite preview answers every unknown path with index.html and a 200, which is why nothing in the suite could tell a real 404 from a soft one. (5) The home card said « the three modules are live » — written at V22, untouched through eighteen waves. It is now « What's new »: a current map of the five areas, the latest wave read from the build version, a link to the log. (6) A CHANGELOG.md extracted from this file — one entry per wave from V7 on, the 27.x honesty fixes included and V12 (pending) excluded, newest first — by npm run changelog, which also aligns package.json on 1.<wave>.0; a test compares the committed file with the generator's output byte for byte, so the changelog cannot drift from the plan in either direction, and pins the version to the last wave recorded here. Three rows of this file (V25–V27) turned out to have no « why » cell; they are kept with an empty rationale rather than dropped. The V32 row held an unescaped | inside inline code, which split it into four cells and made its published rationale a fragment of its content: the pipe is now escaped, and the generator refuses any wave row with more than three cells — or with a status it does not know — rather than skipping it, because a row skipped in silence leaves the version a wave behind with every test green. Also: the « PS: » about dataset size leaves the/ml lede for a note under the drop zone, where the file arrives, and the README's duplicated LIMITATION line is one line. What this wave deliberately does not do: translate the changelog (the wave titles are this file's, in English — the card shows the number), prerender the documentation in French (the shells are English like every section; the app switches on mount), or add hreflang alternates (the language is the app's, not the URL's). 817 unit tests, 122 e2e. |
Owner request (29/09/2026): an analysis of the repository and of the production site, then a first wave. All six items are defects in what the site said about itself rather than missing features — a lab that publishes its refusals and its limits cannot have its documentation indexed as twelve copies of its home page, nor answer 200 to an address that does not exist. |
| V42 — delivered | A scientific reference outside our own implementation. The metrics and the simplest compatible models were tested against hand-calculated examples, but a hand-written engine checking itself can reproduce the same misunderstanding in both code and test. A Python generator now computes committed references with scikit-learn 1.7.2 for multiclass accuracy, macro precision/recall/F1, log-loss, binary ROC-AUC with ties, RMSE, MAE and R². It also records predictions from two algorithms whose assumptions can be made identical rather than merely similar: ridge regression with the same regularized bias column, and five-neighbor classification/regression with uniform Euclidean votes. Vitest consumes the JSON fixture in ordinary CI, so Python and scikit-learn are not build dependencies; the generator runs only when someone deliberately reviews and regenerates the reference. The fixture records its Python and scikit-learn versions, and the test refuses an empty fixture so a missing generation cannot turn numerical checks into vacuous passes. Forests, trees, GBDT, MLP and logistic predictions are deliberately excluded: their implementations or defaults differ, and demanding identical numbers would freeze a false equivalence rather than catch a scientific regression. 830 unit tests, 122 e2e. | This is the quality contract promised in §F since the original architecture. Independent expected values catch a wrong formula even when the local unit test repeats the same wrong assumption; committing the small fixture gives CI that protection without adding a Python toolchain to the application. |
| V43 — delivered | A failed section no longer takes the whole lab with it. Two React error boundaries now separate failures by reach: a route feature can stop while the header, navigation and footer remain usable; a second boundary catches failures above that shell. Both offer a retry without forcing a reload and a direct path home. React Router has its own internal error boundary, so two matching route boundaries prevent a failed lazy import or route resolution from falling through to its default page, which prints the error message and stack; those failures offer an explicit reload because a rejected module is cached and cannot be retried by resetting React state. Navigating to another section resets the local boundary instead of leaving the user trapped on the failed view. The recovery screen is bilingual, keyboard-accessible, takes focus when it replaces failed content and is explicit about what happened. It can copy a short technical report, but only after the user asks: app version, timestamp, route pathname, language, browser, a validated error class and at most twelve sanitized, length-limited React component names. Every caller-controlled field is length-limited and stripped of control characters. The report deliberately excludes the error message and JavaScript stack — either can contain a cell value or file name — as well as URL parameters, fragments, dataset metadata and absolute source paths. If clipboard access is denied, the same report remains visible and selectable. Nothing is transmitted automatically, preserving the privacy claim rather than weakening it in the name of diagnostics. What this wave deliberately does not do: catch errors inside event handlers or asynchronous callbacks (React boundaries cannot), add telemetry, upload crash reports, or persist reports locally. 841 unit tests, 122 e2e. | The application already protected data at the network boundary, but one unexpected render failure could still replace the complete interface with a blank page and leave no safe diagnostic to share. Recovery now follows the same rule as the rest of LabML: preserve useful context, disclose the limit, and never move data without an explicit action. |
| V44 — delivered | The supply chain now fails closed where the signal is actionable. Dependabot checks npm and GitHub Actions every Monday; minor and patch updates for development dependencies and Actions are grouped, while major updates stay isolated so their migration cost remains visible. CodeQL analyzes JavaScript and TypeScript on every pull request, every push to main, and weekly, with only contents: read and security-events: write. The quality job now audits the complete installed tree for critical advisories and the production tree for high advisories. That split is deliberate rather than lenient: the initial measurement found 21 high advisories, most inside Lighthouse, Puppeteer, Wrangler and their command-line dependencies, including several whose only npm remedy is the destructive downgrade @lhci/cli@0.1.0; blocking every PR on those would turn the control off in practice. Production was different: four high advisories came through Transformers' Node-only companions (adm-zip and Sharp), even though the browser does not load them. Transformers stays pinned on 4.2 because 4.3's WebGPU runtime contains two indivisible WebAssembly modules above Cloudflare Pages' 25 MiB file limit; npm overrides update only onnxruntime-node and Sharp, removing every production advisory without changing the browser runtime or the already-sharded model weights. The enforced production audit now reports zero. Four committed policy tests pin the ecosystems, cadence, update grouping, CodeQL triggers/actions/permissions, both audit thresholds and the rule that Dependabot PRs never request unavailable deployment secrets, so the configuration cannot quietly weaken. What this wave deliberately does not do: run Dependabot daily, group major upgrades, apply npm audit fix --force, or pretend that high-severity findings in build tools do not exist — Dependabot keeps their fixes moving, CodeQL covers application code, and any critical advisory anywhere still blocks CI. 845 unit tests, 122 e2e. | The plan promised Dependabot, CodeQL, npm audit and least-privilege Actions permissions in §E but none was active. A security gate is useful only when it distinguishes a shipped exposure from an advisory with no safe upgrade path; this wave measures the baseline, removes the production findings, and makes future regressions blocking. |
| V45 — delivered | A complete multiclass decision rule — including the right not to decide. V36 deliberately stopped at one-vs-rest diagnosis: it can show the precision-recall and calibration curves for one class, but it cannot turn several class thresholds into one answer. V45 closes only that gap. A policy holds one minimum probability per class; zero means « no minimum » and reproduces the model's current argmax behavior. For each row, classes that clear their minimum are eligible. One eligible class wins directly; when several qualify, the largest normalized excess (probability - threshold) / (1 - threshold) wins, with the stable class order breaking an exact tie; when none qualify, the result is an explicit no decision, never a silent fallback to argmax that would make the thresholds decorative. The editor works from the validation split, not the held-out test: it shows coverage, overall accuracy with abstentions counted as errors, accuracy among decided rows, macro precision/recall/F1, and the same metrics per class, all beside the untouched argmax baseline. The rule is frozen before the test result is revealed; changing it afterwards marks the result exploratory rather than presenting a tuned test as an unbiased estimate. The chosen policy joins the persisted run, comparison and report, then applies to batch scoring: exports keep the raw argmax and per-class probabilities, add the policy decision and an explicit decided/abstained status. Model export moves to format v4 with an optional decision policy; v2 and v3 remain importable, and a v4 file without a policy behaves exactly like today's model. Unit tests pin eligibility, normalized arbitration, ties, abstention, baseline equivalence at zero, metrics and serialization; Playwright covers editing, freezing, the honest test-state transition, persistence and batch CSV. What this wave deliberately does not do: choose thresholds automatically, hide a full misclassification-cost matrix behind one slider, call abstention uncertainty calibration, promise conformal guarantees, or alter binary threshold behavior. 871 unit tests, 122 e2e. | Approved 02/10/2026 after the Dependabot hygiene pass. The current multiclass panel is honest about its limit but leaves the user with several disconnected one-vs-rest readings. A deterministic policy, an explicit abstention and a validation-before-test workflow turn those readings into an actionable decision without pretending that every row deserves an answer or spending the test set while tuning the rule. |
| V46 — delivered | Five waves of documentation debt, paid in one pass. /docs/formats published an export manifest that no longer matched serializeModel, and nothing on the public site explained the multiclass decision policy shipped in V45. Four page pairs are rewritten against the code rather than against the previous wave's prose: the real v4 manifest with its conditional decisionPolicy block and the five named import refusals; a reference section on deciding across several classes — one threshold per class, normalized-margin tie-break, explicit abstention, all-zero thresholds reproducing argmax; the policy_decision and decision_status columns a scored CSV now carries, and what an empty cell means for the row it describes; and four renunciations from V41, V43, V44 and V45 recorded in the limits pages. The README names the policy in its imbalance row. What this wave deliberately does not do: rewrite the tutorial, add a page, translate the changelog, quote a figure that is not traceable in this plan, or touch a single source file under src/features/. 871 unit tests, 122 e2e. | Approved 03/10/2026, right after V45 merged. A reference page describing a format the code does not produce is worse than a missing page, because it is read as authoritative. The fix is not to paraphrase the wave entries but to read serialize.ts, deserialize.ts, score.ts and multiclass-decision.ts and write down what they actually do. |
V30's measurements, all on the same 55 questions, the same pinned q4f16 weights and the same CPU runtime. « app » is what a visitor actually gets: the deterministic parser first, the model only on what it refuses.
| configuration | model alone | app |
|---|---|---|
| V27.3 as shipped — free decoding, frozen examples | 29 / 12 / 14 | 33 / 15 / 7 |
| V30 parser alone, no model at all | — | 19 / 0 / 36 |
| + constrained decoding, frozen examples | 34 / 19 / 2 | 45 / 9 / 1 ◊ |
| + generated examples, first version | 32 / 12 / 11 | 34 / 11 / 10 |
| + the aggregate-with-filter shape | 34 / 10 / 11 | 37 / 9 / 9 |
| + examples on a quantity, not a flag — shipped | 41 / 7 / 7 | 42 / 7 / 6 |
| the shipped prompt, decoded FREE (CI, another machine) | 34 / 8 / 13 | 37 / 7 / 11 |
Read as right / wrong / refused. ◊ That line is not comparable: its prompt contains seven of the 55 questions verbatim, so on those it measures recitation.
The last line was not planned — it is what the first CI run of the bench
produced, because the constraint was opt-in and the workflow did not opt in.
A bench that measures a configuration nobody ships is worse than no bench: a
wrong number with a green tick beside it. The flag now points the other way
(LABML_LLM_FREE=1 to turn the constraint off) and both the browser and the
bench read one shared constant, with a test that fails if they drift. And the
accident paid for itself: it isolates what constrained decoding is worth once
the examples are right — the same prompt, free vs constrained, is +5 right,
−5 refused and no more wrong answers at all. Against V27's frozen examples
the same change had cost +7 wrong. Constraining the output only turns refusals
into confident errors for shapes the model has never been shown; the fix for
that is the example, not the constraint.
It is listed because it is the number the wave would have reported if nobody had
checked, and because it is the only configuration with fewer refusals and more
wrong answers than the one that shipped — which is the trade constrained
decoding makes when refusal is not left reachable.
Latency is deliberately absent from that table: it was measured on a CPU under concurrent load, which says nothing about a visitor's WebGPU run. What was measured in isolation is the mask itself — 16 ms at its most expensive step, about 1 ms at every other — and that cost does not shrink on a GPU.
And the owner's question, answered with the instrument rather than an estimate. The bench takes the model as a parameter, so the same 55 questions were put to Qwen3-1.7B q4f16 — 1.43 GB, four times the download, on the same CPU runtime, with the same prompt and the same constrained decoding:
| model | download | model alone | app |
|---|---|---|---|
| Qwen3-0.6B-DQ — shipped | 355 MB | 41 / 7 / 7 | 42 / 7 / 6 |
| Qwen3-1.7B | 1.43 GB | 39 / 12 / 4 | 40 / 12 / 3 |
Four times the bandwidth buys five MORE wrong answers. The bigger model is
genuinely better at the questions that are hard for a small one — it reads all
four grouped comparisons (« did women pay more than men? », « survival rate by
sex ») that the 0.6B refused. It is worse at the easy ones: it answers « count
the passengers older than 60 » with aggregate op=count column=age, which
counts the rows that HAVE an age rather than the rows, and it reads « average
age of women » as a group-by rather than a filter. Net, on this task, it loses.
Two limits, stated rather than buried. The prompt was tuned against the 0.6B — these examples, this refusal example, this choice of measure column were all settled on the small model's failures, and a prompt fitted to the 1.7B would plausibly score differently; the number above is a property of this pair, not of the model. And it is one rung: 764 MB (gemma-3-1b-it) and 1.09 GB (Llama-3.2-1B) were not tried, both because of licence terms stricter than the Apache-2.0 the rest of LabML ships under, and because the rung that was tried went the wrong way. What the measurement does settle is the shape of the decision: « bigger » is not a direction of improvement here, it is a trade whose sign has to be measured — and now can be, in one command.
V31's measurements, all on the same fourteen verified images, replayed in the real browser through the real canvas and the real worker. Read the last three columns as: how many images with no possible ImageNet label were refused, how many wrong labels were announced as wrong, and how many CORRECT answers a refusal cost.
| configuration | top-1 | refused / 4 | caught / 1 | cost |
|---|---|---|---|---|
| V23 as shipped — always answers | 9 / 10 | 0 | 0 | — |
| + rule 1 alone (person AND face) | 9 / 10 | 3 | 0 | 0 |
| + rule 2, floor at 50 % — shipped | 9 / 10 | 4 | 1 | 0 |
| the same rules on a full-frame squash instead of the crop | 9 / 10 | 4 | 1 | 0 |
The last line is the refutation of this wave's own hypothesis about the crop. It is also where the argument for rule 1 comes from: under the squash the photograph of people under an umbrella is classified « umbrella » at 99.3 % — plausible, confident, and not what the picture is of. A confidence floor alone, at any value that does not also refuse most correct answers, passes that one straight through.
The detector's two misses were probed rather than assumed. With
OBJECT_THRESHOLD dropped from 0.35 to 0.05, YOLOX-Nano still produces no
sports ball box on the football image — its best candidates there are
« toilet » at 14 % and « toaster » at 8 % — and no bottle box on the wine
image. Neither miss is a calibration problem, so neither is fixed by moving the
bar, and moving it down would admit exactly the kind of noise the 0.35 value was
chosen in V23 to exclude. The same probe is what justifies rule 1's face
requirement in numbers: at 0.35 the wine image carries a false person box at
38 %, and the sports car another — both silent, because UltraFace finds no
face on either.
One limit, stated rather than buried. Fourteen images is a narrow base. The 50 % floor sits in a gap between 41.2 % (the dearest wrong answer rule 1 does not already take) and 70.5 % (the cheapest correct one) that is wide but weakly sampled: a fifteenth image could land inside it, and the first one that does will cost either a correct answer or an honest refusal. The floor is a named constant with the measurement written beside it, so the next corpus can move it rather than discover it.
V37 — parallel training and the k-NN bottleneck. Same machine, same 60 000-row file, same seed; four arms.
| k-NN as shipped before | k-NN fixed | |
|---|---|---|
| sequential | 73 950 ms | 14 654 ms |
| parallel | 69 738 ms | 9 509 ms |
V38 — the same file, two spellings. Same 900 rows, same three numeric columns, same seed; only the way a number is written changes.
| written | inferred types | features after encoding | champion |
|---|---|---|---|
12.50 |
numeric × 3 | 3 | 1.000 |
12,50 |
text × 3 | 103 | 0.819 |
Ordering: V38 came before V39 and V40, and for the same reason V35 came first in its own group: its headline item is a defect in shipped code, not a feature — a studio that promises honest data cannot silently turn 12,5 into NaN. V35, V36, V37 and V30 are delivered. V32 ships one finished tutorial before any reference page — the tutorial is the template the rest copies, and settling it late means rewriting everything. V30 is delivered, and it started with a bench for exactly the reason written here: « a bigger model » was not a measurable statement. It is one now — the bench takes the model as a parameter — and V31 still has its own instrument to build. No wave starts without an explicit launch command. V23 first (owner request); V24 keeps its vocabulary capped —
V25 (delivered) chose announced sampling and a named memory guard over the
typed-array rewrite, which measurement showed unnecessary; widening the
vocabulary stays possible later. Set aside for now: multiclass thresholds. V12
(consented generative chat) remains pending a product decision — no wave
starts without an explicit launch command.
Cap 6 recap: V23 to V46 delivered, the last one on 03/10/2026. The « set aside for now: multiclass thresholds » line above is out of date: V45 shipped them. V12 (consented generative chat) still waits on a product decision.
Why Cap 7 exists. An audit of the shipped code on 09/10/2026 found the
suite green (lint, types, format, 872 unit tests, npm audit clean) and
still found defects that the suite could not see, because none of them lives
inside one function: they live in the conversation between a store and its
worker. The worst one wrote wrong data. Train on titanic, change the target
while the run is still going, and the auto-save recorded the new target beside
the old scores. Cap 7's guiding thread follows from that: before the lab
learns anything new, what it already does must stay true when the user does
two things at once, when a worker dies, when a link is hostile, and when CI
runs on a branch name nobody checked.
Ordering rule, the same one V35 and V38 applied: defects in shipped code come before features, and the one that produces wrong data comes first of all.
| Wave | Content | Why |
|---|---|---|
| V47 — delivered | Workers that cannot answer the wrong question. Six lifecycle defects in shipped code, each reproduced by a test that failed before its fix. (1) A result filed under a target it was not trained for: setTarget and toggleColumn reset the training state without cancelling the run, the worker kept posting model-result, and the auto-save recorded the new target beside the old scores. The store now refuses both changes while anything computed for the current question is running (training, tuning, learning curve, robust ranking, batch scoring), the target picker and the column buttons are disabled for the same span, and a fence (isStaleResponse) drops any training message outside a run and any run-derived answer when no run is done; the worker answers in order, so the training status is enough to tell a late answer from a current one, without a run id in the protocol. (2) Cancel did not reach the parallel phase: V37's helpers were only checked after every one of them had finished, so cancelling waited for the heaviest families; the phase now polls the flag every 100 ms and terminates the helpers mid-fit. (3) A crashed ML worker left trainStatus at training forever and kept the dead reference; onerror now terminates it, clears every busy flag, and the next request starts a fresh worker. (4) The same crash in the chat left « thinking » up and still offered a local model that died with its worker. (5) DuckDB was never closed: each visit to the SQL panel left a worker and its Wasm heap behind, a failed start leaked its worker too, and a failed Parquet export left its file in the heap; the panel now closes the engine on unmount, on a late start and on failure, and export names come from a counter rather than a millisecond clock. (6) A webcam stream granted after the page was left kept the camera lit; it is now stopped on arrival. (7) Found by this wave's own CI: touching the chat ran the interpretation bench, which failed on size-mismatch:tokenizer_config.json. prepare-llm.mjs called its revision « pinned » while it held main, and upstream commit 1e94c62 (07/10/2026) had grown that file from 9 762 to 10 416 bytes; the revision is now the commit the sizes were measured on (d6beae7), which also keeps the next deploy cache miss from failing the same way. What this wave deliberately does not do: add a run id to the worker protocol (the order guarantee makes it redundant today, and it is the first thing to add if requests ever become concurrent), cancel analyses the user cannot cancel today, or touch the deeper share-link validation, which is V48. 883 unit tests, 123 e2e. |
Approved 09/10/2026 after the post-V46 audit. The defect in (1) does not crash and does not warn: it saves a run that looks valid and is not, which is the one kind of failure an honest lab must never produce. The others are the same class of bug, a worker outliving the state it was started for, and fixing them together keeps the rule in one place. |
| V48 — delivered | A damaged share link ends on a refusal, never on a crash. The plan for this row was a deeper validator: check insights.importance, numeric metrics, classes and confusion, the fields the audit had named. The wave opened with a measurement instead, and the measurement changed the plan. Three real links were captured from the running app (titanic with uncertainty, threshold and segments; iris with k-NN tuning and groups; mpg as a regression) and every field of each was deleted, nulled, or retyped as a string, a number, an object or an array: 3 948 damaged links, 3 660 accepted by the decoder, 1 020 of those crashed the page, on 99 distinct paths, from insights.importance to artifacts.exploration.clusters[].traits[].clusterMean. A validator for 99 paths would be the largest file in the share module and would drift with every analysis a run learns to carry. So the decoder keeps checking what every view reads first, and SharedRunPage wraps the run in a render guard: any failure while drawing it shows the same refusal page, with its own explanation (the link decoded, but part of the run does not have the shape LabML writes) and nothing from the run, so no half-read figure can pass for a real one. The three links are committed under tests/share-links/, and the measurement is now a test: every field shape is damaged six ways and the page must end on the run or on the refusal; it fails on the code before this wave and adds about 25 s to the unit suite, the price of covering the whole format rather than a sample. The five copies of the download helper become one, src/lib/download.ts, which revokes its object URL after the click has been handled instead of in the same tick, where some engines cancel the download silently. What this wave deliberately does not do: write a field-by-field schema for every artifact, show a partially readable run, or guard event handlers (a click on a damaged run is not reachable: the refusal replaces it first). 889 unit tests, 123 e2e. |
Approved 09/10/2026 after V47. A link is input from a stranger, and the most common stranger is a chat client that cut it. The decoder already refused the shapes the V35 audit had found; the measurement showed it could not keep up with the format, and that the robust answer was at the point of failure rather than at the gate. |
| V49 — delivered | CI that cannot be steered by a branch name. The wave opened with zizmor 1.30.1 on the three workflows: 28 findings (1 template injection, 21 actions pinned by a movable tag, 6 checkouts leaving the git token on the runner disk). (1) Injection: the deploy step expanded github.head_ref inside its script, so a branch named x";…;" would have run as code; the value now reaches the shell through env:. Low severity today (only branches of this repository deploy) and fixed before that stops being true. (2) Pins: every action is pinned to a full commit SHA with its release in a trailing comment (# v7.0.1), which Dependabot keeps current; a tag can be moved by whoever controls the action's repository, a SHA cannot. (3) Credentials: every checkout sets persist-credentials: false; no job pushes, so nothing needed the token on disk. After the fixes the offline audit reports 0 findings, and a new Workflow audit (zizmor) job runs it on every pull request, installed from PyPI at a pinned version rather than through a third-party action, with the token the online audits need to check that each pinned SHA really belongs to its repository. Three new tests in supply-chain.test.ts hold the same rules per workflow file (SHA plus release comment on every uses:, no persisted credentials, no pull-request-controlled ${{ }} inside a run:); they fail on the workflows before this wave. (4) Coverage: npm run test:coverage (V8 provider) replaces npm run test in CI with a floor set from the measured baseline rounded down, not a round number: 64.56 % statements, 53.28 % branches, 54.46 % functions, 65.62 % lines on 898 unit tests, floors 64 / 53 / 54 / 65. The figures cover unit tests only; the interface is mostly exercised by the e2e suite, which this does not measure. Cost: the quality job grows by about a minute. What this wave deliberately does not do: chase a coverage target, add e2e coverage instrumentation, or gate deploys on the audit job. 898 unit tests, 123 e2e. |
Approved 09/10/2026 after V48. Continues V44: a security gate is worth its maintenance only when it distinguishes a real exposure, and every rule here is now enforced by a test or a job rather than by a reviewer remembering it. |
| V50 — delivered | Metrics that agree with the reference on every edge. Measured before anything changed, then checked against scikit-learn rather than against our own reading of it. (1) Macro averages ran over every class the task knows; scikit-learn averages over the labels present in y_true or y_pred. A perfect score on a batch holding two of three classes read 0.667 instead of 1.0. Both implementations (macroPrf and the V45 decision policy) now follow scikit-learn, and the V42 generator gained an absentClass case (four classes declared, three present, one predicted but never true) that the old code failed. (2) Rare classes: the stratified split sent a class's only row to test, and the validation carve did the same to a two-row class, so the models never saw an example of either. Measured on 100 rows plus a rare class: 1 row gave test 1 / train 0, 2 rows gave test 1 / validation 1 / train 0. A single-row class now stays on the training side (1 → train 1; 2 → test 1, train 1; 3 and more unchanged, and the seeded draw of every other class is byte-identical). This is not hypothetical: three targets the demos offer have one-row classes (titanic → parch, cafe-sales → city, cafe-sales → product). On titanic → parch every model's macro F1 rises once the empty classes stop counting (GBDT 0.619 → 0.716, ensemble 0.422 → 0.548), and the decomposition shows the metric change alone moves nothing while every class is present: the gain comes from the split. The validation-picked winner moves from GBDT to the forest because the validation rows changed, not because a metric did. When every class has one row, nothing is left to test on and training refuses by name (too-few-rows) instead of reporting metrics over an empty set. (3) Empty sets: accuracy, log loss, RMSE, MAE, R² and the macro trio now return NaN on zero rows; before, RMSE 0, MAE 0 and R² 1 described a perfect model measured on nothing. No caller passes an empty set today (the test split keeps a row, batch scoring skips unlabeled files), so this is a contract for the next caller; the one theoretical path left, tuning on fewer than four rows, is noted rather than guarded. What this wave deliberately does not do: change any figure on a dataset whose classes all have three rows or more (every demo target used by a test or a tutorial), add a warning banner for rare classes, or guard k-fold tuning on tiny data. 906 unit tests, 123 e2e. |
Approved 09/10/2026 after V49. The same rule as V35: a number must not flatter or punish by accident. Two of the three edges were reachable from the demo picker, and the reference that would have caught the first one existed since V42 but held no case with an absent class. |
| V51 — delivered | A store that fits on a screen. No behaviour change, and the proof is the suite: the 906 unit tests and the 97 Chromium e2e tests that passed before pass unchanged after. (1) lab-store.ts, 1 050 → 530 lines. The 250-line onmessage chain becomes a dispatch table in store/responses.ts, one entry per WorkerResponse kind, typed as a mapped type so the compiler refuses a kind with no handler and a handler for a kind that does not exist. The chain's last else used to catch every unlisted kind as an error silently; that safety net is now a compile error instead. The bodies were moved by script, not retyped. The state shape and starting values go to store/state.ts, V47's stale-response fence to store/fence.ts, the threshold artifact builder to store/artifacts.ts; lab-store.ts keeps the actions and re-exports its public names, so no importer changed. Feature slices were considered and not done: the actions share one worker and one send, and slicing them would have moved code without separating anything. (2) Tests named after waves renamed after what they pin: v35 → honest-evaluation, v36 → imbalance-ensemble, v37 → knn-and-helpers, v38 → french-numbers, v39 → column-recipe, v40 → quality-score. The two mixed files (v35, v36) were renamed, not split, to keep the change mechanical. (3) Duplicated helpers: numbersOf, missingRatio and the binned shares, copied identically in drift.ts and reference.ts, now live once in quality/distribution.ts. The two quantile functions were left apart on purpose: on an empty list one returns 0 and the other NaN, and merging them would have been a behaviour change. Measured side effect: the coverage gate from V49 fired. The 35 branches became 35 functions, so the function denominator rose from 2 142 to 2 174 while covered functions barely moved, and the floor of 54 % failed at 53.72 %. Rather than lowering the floor, a table-driven test (store/responses.test.ts, 17 cases) now pins which fields each state-only answer writes, which is also the check a moved body needs; function coverage is back to 54.46 %. What this wave deliberately does not do: change a single handler body, split the mixed test files, or unify the quantile functions. 923 unit tests, 123 e2e. |
Approved 09/10/2026 after V50. Only after V47 to V50, as the plan required: the store's concurrency rules and the metrics were pinned by tests before their code moved, so a regression in the move would have failed loudly. |
| V52 — delivered | Major upgrades, one at a time, each measured. Three majors, three pull requests, each with the before/after numbers that matter for it: Vitest 5 (V52.1), vite-plugin-pwa 2 (V52.2), TypeScript 7 (V52.3, measured and deferred). Two shipped with nothing moved but rounding; the third passed on our code and was held back by the ecosystem, which is a finding, not a failure. | V44's rule: a major update keeps its migration cost visible. |
| V52.1 — delivered | Vitest 5, measured against Vitest 4 on the same commit. The first of V52's three majors, each in its own pull request. The migration guide was read first and the repository searched for every pattern it names: no toThrow(''), no sequential, no bench, no removed vitest/* entrypoint, every vi.mock already at the top level of its file. Two changes could still have moved results silently, and both were measured rather than assumed. clearMocks now defaults to true: the 924 unit tests pass unchanged on the first run, so no test relied on calls accumulating across tests. Coverage paths are now matched relative to the root: the four figures move by rounding only (statements 64.93 → 64.92 %, lines 66.06 → 66.05 %, branches and functions identical; five statements fewer in the denominator), every V49 floor still met. Timing on the same machine stays inside its own noise: three coverage runs on Vitest 4 took 124 to 143 s, three on Vitest 5 131 to 134 s; plain runs 82 to 87 s on both. Node: Vitest 5 requires Node 22.12, so engines moves from >=20 to >=22.12 and the README says so; CI and .nvmrc were already on 22. What this sub-wave deliberately does not do: adopt Vitest 5's new options (sharedViteServer, artifact directory) or touch the other two majors. 924 unit tests, 123 e2e. |
Approved 09/10/2026 with V52. The safest of the three majors goes first: the suite it runs is also the instrument that proves it changed nothing. |
| V52.2 — delivered | vite-plugin-pwa 2, and a service worker that did not change by one byte of content. The second of V52's three majors. The 2.0.0 release notes list two consumer-facing changes: Node 20.19 as a floor (already covered by V52.1's 22.12) and @vite-pwa/assets-generator 2 accepted as a peer (not used here). A service worker breaks late and on someone else's machine, so the output was compared rather than trusted. Built on the same commit with 1.3.0 and 2.0.0: 98 precache entries, 5 675.73 KiB, in both; the same 98 URLs; sw.js identical once revision hashes are masked; the same Workbox runtime file down to its content hash (workbox-2ff6bd68.js); the web manifest byte-identical. The offline e2e test (the whole lab working after the first visit, network cut) and the update-prompt unit tests pass unchanged. What this sub-wave deliberately does not do: change any PWA option. 924 unit tests, 123 e2e. |
Approved 09/10/2026 with V52. The one upgrade of the three whose failure would reach visitors rather than developers, so its proof is the artifact visitors receive. |
| V52.3 — delivered | TypeScript 7, measured and deferred. The third major, and the one the plan flagged as uncertain. TypeScript 7 is the compiler rewritten in Go. Measured on the same commit, tsc --noEmit on the whole project: TypeScript 6.0.3, 10.8 to 12.3 s over five runs; TypeScript 7.0.2, 1.5 to 1.7 s warm (3.9 s on a cold first run); 0 errors from either. Our code is ready. The ecosystem is not: typescript-eslint 8.71.1, the latest release, declares typescript >=4.8.4 <6.1.0, and the lint is part of the CI gate. Two ways forward were weighed and the simpler one taken. Keeping TypeScript 6 for the lint and adding TypeScript 7 under an alias for the typecheck alone would save about ten seconds on a quality job of three and a half minutes, for the cost of two compilers to keep in step; not worth it. So TypeScript stays on 6, Dependabot ignores typescript >=7.0.0 (a weekly pull request that can only fail the lint is noise, not a signal), and supply-chain.test.ts holds that rule like the others. Resume condition: typescript-eslint supports TypeScript 7; on our side the migration is then a version bump, since the code already passes. Recorded on the limits page as deferred after measurement, as « about seven times faster » rather than in seconds, since the docs quote no wall-clock duration. What this sub-wave deliberately does not do: disable or weaken the lint to make room for the faster compiler. 924 unit tests, 123 e2e. |
Approved 09/10/2026 with V52, option chosen by the owner after the measurement. A gate is not removed to make an upgrade fit; the upgrade waits for the gate. |
Cap 7 closed (09/10/2026): V47 to V52 delivered. What the lab already did now holds when the user does two things at once (V47), when a link is damaged (V48), when CI runs on a branch nobody checked (V49), when a class is rare or absent (V50); the store fits on a screen (V51), and the three majors were taken one at a time, one of them deferred on a measurement rather than forced (V52).
V12 (consented generative chat) is not scheduled: the owner confirmed on 09/10/2026, once Cap 7 closed, that it stays postponed. The row keeps its « pending » status because nothing was built or refused; it is a decision not to start, not a measurement. No wave starts without an explicit launch command.
Written at the close of Cap 7 so the next session starts from a list rather than from a fresh audit. None of these is a wave: each has the condition that would justify starting it, and none starts without an explicit launch command.
- Adopt TypeScript 7. Condition: typescript-eslint supports it (V52.3). The code already typechecks with 0 errors; the work is a version bump, removing the Dependabot ignore, and re-measuring the typecheck time in CI.
- Re-measure DuckDB-Wasm above 1.28.0. Condition: a release whose eh and mvp binaries fit under Cloudflare Pages' 25 MiB per-file limit, or a reason to shard them (V29). The Dependabot ignore keeps the question closed until someone re-measures; it should be re-opened at least once a year.
- Raise the coverage floors where they are lowest. Functions (54 %) and branches (53 %) trail statements and lines. Condition: none, it is maintenance; the method is V51's, tests on the modules with the lowest function coverage, then the floor moves up and never down.
- A lint with zero warnings.
badge.tsxandbutton.tsxexport their variant helpers next to the component, whichreact-refreshflags. Moving the helpers to their own file and adding--max-warnings 0makes the next warning a failure instead of a line nobody reads. - A run id in the ML worker protocol. Condition: the day two requests can be in flight at once (for example, an analysis that runs during training). Until then V47's fence is enough, and the limits page says so.
- Widen the text vocabulary. V24 capped it; V25 measured that the pipeline was never the bottleneck, so the typed-array rewrite was dropped and a named memory guard was added instead. Condition: a real dataset where the cap visibly costs accuracy, measured before and after.
- V12, consented generative chat. Postponed by the owner (09/10/2026). Condition: a product decision, plus a fresh read of the provider's data policy, since the consent screen would have to quote it.