v0.13-supervised-optimization.md

CopperMCP v0.13: supervised autonomous routing slice

• Status: integration in progress; native vertical slice implemented, release gates not measured
• Baseline: 40db464939a89f45feefbf067a750ffc3e06a97b (package version 0.12.0)
• Scope owner: maintainer / release reviewer
• Contract: optimization/v1 draft
• Decision: ADR-0132

Product promise and boundaries

The intended workflow is:

immutable board snapshot -> placement candidates -> internal/external routing -> validation/DRC -> judge evidence -> bounded repair -> selected package -> human review

This cycle is a supervised autonomous routing slice, not a completed electrical-engineering judge. The coverage target is ordinary 2–8 layer KiCad boards with common tracks, arcs, through-vias, zones, keepouts and custom pads, subject to an explicitly published supported subset. Unsupported constructs must produce typed non-success outcomes, not be silently erased, flattened or excluded after an attempt fails. Coverage is a release target, not present evidence.

apply_candidate and apply_placement_candidate remain the only board-writing surfaces. Both stay default-off behind the operator setting and require their own operation-specific single-use token. The optimization coordinator does not invoke them. No live-editor mutation, automatic placement-plus-routing batch apply, or fabrication approval is added by this plan.

Foundation delivered in this change

src/copper_mcp/optimization/ contains closed immutable request/command models, composed-candidate and judge metadata, a pure compare-and-swap lifecycle, cumulative resource reservations, and a bounded process-local human-confirmation capability primitive. The integration adds a redacted SQLite repository, fenced worker, cumulative budgets, five closed MCP wrappers, private native placement/routing composition, repeated KiCad DRC and DRC-backed DFM, and optional bounded Circuit Intent ERC. Full candidate-board resources require separate trusted-host disclosure consent, expire after five minutes, and are revoked when their job is cancelled. Existing routing jobs and placement search are reused rather than replaced.

The native path supports the underlying router/serializer subset, not arbitrary boards. Zoned compositions refuse until candidate-bound fresh fill is implemented; cross-layer multi-pin trees, production DSN/SRJ conversion, normal workspace schematic ERC, before/after rendering, bounded repair coordination, and Orca advisory scheduling remain work in this cycle. The container runner and recipes do not by themselves implement either external format bridge.

The current soft congestion metric is bounded coarse occupancy of straight tracks and through vias. Clearance headroom is a conservative zero lower bound, not a measured optimum. Actual native routing and mandatory DRC precede selection; Manhattan remains a search heuristic.

scripts/check_optimization_release.py validates a submitted evidence checklist and the numeric gates below. It cannot authenticate the referenced artifacts. Even a complete checklist returns artifact_authenticity_verified: false and release_authorized: false; its best outcome is eligible_for_independent_artifact_review. It cannot publish, tag, or write a board.

Tests for the foundation use synthetic observations explicitly labelled as such. A synthetic pass tests a binding invariant, not a real router, KiCad result, corpus metric or human channel. The package version stays 0.12.0; no release-ready ledger row or empirical benchmark is added.

Week 1: freeze contracts and perform preflight

The registered wrappers resolve private workspace inputs into the foundation's internal command contracts. They execute over stdio only until authenticated network ownership is implemented. Host confirmation is the selected interface; clients that auto-answer elicitation must not be configured as a trusted human channel. COPPER_MCP_OPTIMIZATION_HOST_CONFIRMATION remains off by default. Package approval does not approve source-file mutation.

Preserve the original dirty checkout and work in the isolated integration worktree. PR #267 is the foundation, PR #259 supplies portable B-141 provenance, and PR #265 supplies composed-bundle DRC. Reuse relevant PR #263 placement service/testing work without adding a standalone public solver. PR #266 is measurement-only; the live-apply design stays outside this release.

Start the licensed corpus intake and family-level held-out split in this phase, before tuning. The selected external deployment is an operator-installed local Docker-compatible Linux engine; never fall back to an uncontained native external process when it is absent.

The five operations retain these requirements:

Operation	Required behavior of the eventual transport
start_optimization	Parse the closed request; resolve and snapshot an explicitly authorized workspace board; derive caller context server-side; compare source and Board IR digests; validate frozen profiles and backend availability; enqueue without adding apply authority.
get_optimization_job	Require authenticated caller context; return only the redacted retention projection. A content-addressed job ID is an idempotency key, not authorization.
cancel_optimization_job	Require caller ownership and expected record revision; atomically fence the worker; terminate child processes; publish cancellation rather than a partial successful package.
export_optimization_package	Bind disclosure consent to owner, job revision and exact package digest. Export preview metadata for review; disclose geometry only through separately authorized, bounded, ephemeral artifact delivery. No token may be minted by an AI request.
approve_optimization_job	Consume one capability issued by an independent human-confirmation channel for the exact job, revision, package and judge. A model's approved: true, actor name or claimed role is never consent.

CI and source-bound evidence

Quality and the single wheel/sdist/PCM build run on Python 3.12. Compatibility runs product and contract tests on 3.11/3.12/3.13 with four xdist workers and file-level distribution; only 3.12 collects coverage. Evidence replay runs once per selected interpreter, canonical normally and all supported interpreters for sensitive or unknown changes. Nonsensitive PRs omit expensive replay explicitly; main, daily nightly and release retain the evidence path. Package checks reuse the quality-built artifacts after required gates succeed. Static/type/build checks are not repeated in the compatibility matrix.

make test remains serial. test-fast, test-full, test-compat, and test-evidence are additive. Contract tests load the historical self-digesting B-141 artifact, while one marked fresh-process replay pins the complete Python implementation inventory and verifies it unchanged before/after recomputation. Historical companion_bound alone does not prove today's production implementation reproduces the measurement. Preserve all tampering/semantic/provenance checks.

Measure successful PR required-path time below 20 minutes and local fast feedback at least 3x faster than the recorded serial baseline, with selection differences disclosed. Four workers and 120-minute ceilings remain. New job budgets are explicitly provisional; release refuses pending calibration until representative successful main runs replace it. No timing target is measured or met merely by changing the workflow.

The request binds source revision, Board IR snapshot, explicit movable footprint references, placement intent and grid/cardinal rotations, frozen target-net scope digest and target count, routing and judge profile digests, permitted backends, integer objective weights, seed, resource limits, optional electrical input digest, required judge domains and mandatory human review. The side-preservation and approval fields accept literal JSON true, never integer 1. Empty movable scope explicitly means no movement; a later worker must still produce a verified identity placement candidate, rather than silently manufacture one.

Resolve the workspace path through the host's existing bounded workspace boundary. Paths and request-local references stay ephemeral; the durable record stores digests, statuses, aggregate metrics, backend versions, fixed refusal codes, candidate IDs and cumulative counters only. Board bytes, raw geometry, net names, prompts, credentials, command text and solver logs never enter that record. Owner bindings must derive from authenticated, high-entropy host context, not caller-controlled labels or plain hashes of private board names.

Preflight both external engines on the actual deployment host. Record executable/package digest, version, fixed command/profile digest, OS/runtime, input/output digests and the run disposition. Run at least one genuine local smoke for each FreeRouting and SimpleRouteJson/tscircuit. Contract-only tests cannot satisfy these gates. An unavailable executable is a blocker to the completed two-backend hybrid-router claim, not a simulated success.

Orca reconciliation

The baseline already tracks the OrcaRouter provider, tests, ADR-0130 and integration documents. This does not establish that the operator's local /Users/.../12_PCB working tree is clean. Before release, inventory that actual worktree and classify every retained untracked file: commit and document it, or explicitly exclude it from shipping. Scan retained changes for secrets and run the actual sdist tracked-file gate. Do not discard a dirty worktree or assume GitHub contains all local WIP. The plan does not authorize invoking the external Orca application/CLI.

The draft optimization request permits deterministic policy only. A later reviewed adapter may admit OrcaRouter as a disabled-by-default advisory provider with explicit operator configuration; malformed, unavailable and timed-out policy must choose the deterministic fallback. It may select only enumerated net orders, corridors, repair windows and ranking actions. It cannot supply copper geometry, validation verdicts, disclosure consent, human approval or apply capabilities.

Weeks 2–3: production multilayer candidates and hybrid routing

Extend the existing layered candidate path in small, independently reviewed increments. Each increment requires exact ordered layer-stack validation, through-via legality and clearance, source-preserving serialization, combined object/byte budgets, and a full Board IR round trip of the composed derivative. Zone fill is usable only through the existing freshness authority; unknown padstacks, rules, custom-pad topology or source-fidelity cases remain refusals. Blind/buried/microvia policy is outside this release. No promise of arbitrary custom-pad support is made merely because a fixture has the name "custom pad".

Promote negotiated routing to bounded whole-board plans only after it accepts the declared multi-pin/multilayer subset and proves cross-net clearance on the complete composed plan. Rip-up/reroute is transactional candidate work: a failed repair discards its derivative, records a typed non-success and consumes the same global budget. A backend failure or timeout cannot be relabelled a successful repaired run. A separately authorized new job may try another profile.

Implement two isolated adapters. FreeRouting receives a fixed DSN and returns SES in a private bounded work directory with fixed executable/arguments, no shell interpolation, a scrubbed environment, a process-tree deadline, and capped output. DSN export must preserve or refuse KiCad rule/zone/pad semantics rather than silently drop constraints. SimpleRouteJson uses its portable JSON contract and equally bounded local process isolation. Neither adapter may accept caller-selected commands, executable paths, network destinations, or arbitrary plugins.

Normalize both outputs through the existing external-candidate disposer. The coordinator owns identity reconstruction and exact geometry validation; external candidate IDs or success flags are untrusted. Reparse the entire placed-plus-routed derivative and run mandatory candidate-bound KiCad DRC over that same board and frozen rule/library context. Per-net validation alone is not whole-board validation. Version/command/input/output provenance is retained as digests and closed metadata, never raw solver output. Repeat normalized evidence under identical bindings.

Related integration work must be reconciled rather than duplicated: PR #265 develops opt-in composed bundle DRC; PR #266 develops a surrogate/signoff measurement instrument. Their current branches are not presumed merged or accepted. A one-invocation bundle DRC result is not enough for the repeated-agreement judge envelope in this plan.

Weeks 3–4: worker, placement search, package and consent

Build an optimization repository above existing routing jobs, with transactional CAS, worker leases/fencing, bounded record count and retention, and cancellation safe against stale workers. The foundation models the lifecycle, not those database guarantees. Use one global monotonic deadline plus per-phase ceilings. Charge attempts, expansions, probes, bytes and repairs across all candidates; a new phase, backend or repair cannot reset accounting. Future nested workers must reserve and report actual consumption, including failed attempts. Tests must kill workers between reservation, execution and publication.

The success path is queued, inspecting, placing, routing, judging, optionally repairing and routing/judging again, awaiting_approval, approved, completed. Other terminal outcomes include cancelled, stale_revision, budget_exhausted, unsupported_geometry, backend_failure and failed with a fixed cause such as required_domain_inconclusive or interrupted. After losing ephemeral inputs on restart, terminate the run as interrupted unless separately authorized rehydration reproduces all captured inputs; never guess geometry from retained digests. Completed means this approved package workflow finished, not that copper was applied or engineering was signed off.

Call existing solve_placement() only from the optimization worker. Do not expose a second public unbounded placement solver. Explicitly selected movable footprints only, no locked moves, no side changes, supported cardinal rotations and exact grid snapping. Every state passes the existing legalizer and source-fidelity gate. Unsupported courtyards/padstacks/transforms refuse. Rank multiple legalizer-issued candidates using actual bounded route probes, not a Manhattan proxy as final routing-quality evidence.

Retain lexicographic objective tiers: hard legality/DRC first; fully connected target nets; congestion and clearance margin; vias; copper length; displacement and intent residuals; then deterministic candidate-ID tie-break. Positive integer weights operate inside a tier and cannot trade a hard violation for length. The foundation defines metrics/weights, not a new ranking implementation. Final selection happens only after actual routing and judge evidence.

Composed snapshot is mandatory

Bind the package to the original source/Board IR, one placement candidate and its resulting snapshot, one complete route bundle whose base is that placed snapshot, the final composed board revision and rule-context digest. Judge evidence targets this composition, not the original unplaced board or a different candidate. Carry alternate IDs, measured objectives, provenance and every inconclusive/not-run domain. Target count must equal the request's frozen count. The worker must additionally verify target-set membership from the ephemeral scope; a count or scope digest alone cannot prove connectivity. This foundation checks shape/bindings only.

Approval and export are independent capabilities. The foundation's host-owned confirmation primitive is default-off, single-use, capacity-bounded, expires after 600 seconds and is lost on restart. It is not an authenticated UI and cannot distinguish a human from privileged Python code. A future host must own a genuine human channel and transact capability consumption with the durable CAS update; crash/replay tests must prove exactly what is recoverable.

Applying later is deliberately outside the loop. Applying placement changes the revision, so an old route token cannot simply be used as a second step. Re-observe and re-validate/re-preview the route on the new revision, then obtain a fresh separately authorized route token. v0.13 must not claim atomic installation of a combined package. On accepted test copies, run baseline and post-apply DRC with comparable frozen contexts to demonstrate the requested no-regression gate.

Week 4: all-domain judge, without authority inflation

Return exactly DRC, ERC, DFM, SI, PI, thermal and EMC. Each status is pass, fail, inconclusive or not_run. Fail dominates; any remaining unknown prevents all-domain pass. Required-domain status is reported separately: all required domains must have authoritative passes before review, and any failed optional domain blocks selection too. Optional unavailable domains remain visibly inconclusive after human approval. No engineering_signed_off state exists.

Integrate coordinator-owned KiCad DRC; KiCad ERC when valid schematic/electrical intent is available; and a bounded versioned DRC-backed DFM profile. ERC evidence binds the electrical input digest, not PCB bytes. All observations also bind the selected candidate, original board, rule context, settings and executable/backend version. Require at least two agreeing normalized observations before a pass/fail claim; retain disagreement/suppression as non-success, not an arbitrary winner. Do not include clocks or host paths in deterministic judge identity.

Define versioned adapter interfaces for SI, PI, thermal and EMC; until a selected local authority and adequate electrical inputs exist, return inconclusive with a closed reason. The foundation intentionally has no registered physics authority and cannot promote those domains to pass. Domain envelope validation does not authenticate a backend. The future worker must mint evidence only after independently verified invocation; it must not deserialize model-authored "evidence" into approval authority. Existing DFM signoff and its private evidence boundary stay separate; DRC-backed DFM here is not a blanket manufacturability claim or write authorization.

Week 5: general-ordinary-v1 measurement and adversarial validation

Execute the redistributable manifest frozen during intake, with at least 12 open-licensed boards from at least three project families. Include 2/4/6/8-layer cases where available. For every board record upstream source revision, license/SPDX and license digest, immutable source hashes, provenance, family/split, supported-feature inventory and artifact hashes. No private/customer boards or unclear-license downloads enter the corpus. Freeze family-level held-out splits before tuning.

Generate intentionally-unrouted derivatives from captured source bytes only. Record a versioned transformation recipe: exact permitted object classes removed, retained planes/zones/pads/rules, source/derivative hashes and generator version. No original file is overwritten. Validate a no-op replay and pre/post source hashes. Treat derivative validation as its own gate.

Freeze net eligibility, profiles, objectives, seeds, resource ceilings, excluded constructs and stop rules before the held-out run. Runtime failures remain in the eligible denominator; zero eligible targets is undefined and blocked, never 100%. Publish exclusions and coverage alongside success rate so refusing hard boards cannot manufacture a quality improvement. A net counts fully routed only after composed connectivity and mandatory candidate DRC pass; a partial tree or incomplete bundle never counts as success.

Mandatory release evidence: at least 90% of eligible held-out targets fully routed; zero unverified clearance claims; zero hard post-candidate KiCad DRC errors; no post-apply DRC regression on accepted copies; identical deterministic candidate/judge digests; both genuine external smoke runs; no hard placement regressions and improved measured route objective on at least three held-out placement cases; missing-domain honesty; and adversarial proof that AI cannot bypass validation, disclosure, approval or apply. Include cancellation, crash recovery, TTL, malformed outputs, stale snapshots, unsound round trips and side/lock preservation tests.

The checklist conservatively blocks an unqualified 2–8-layer claim without all four layer counts. Where 6/8-layer redistributable sources are unavailable, record the shortage and make an explicit reviewed coverage/scope decision; do not silently turn availability into a passing coverage number. No such exception or evidence has been recorded by this foundation.

Week 6: release decision

Run make check on the exact clean release candidate, including packaging, docs, ledgers, security/dependency audit, types and build. Independently inspect the actual artifacts referenced by the checklist and their commit/corpus/profile bindings. Reconcile any open branch ID collisions, local Orca WIP, server_info and documentation. Only then add the existing release ledger authorization and tag through the established release process. The checklist is a necessary preliminary gate, not sufficient authorization.

Run the preliminary checklist with PYTHONPATH=src python scripts/check_optimization_release.py /path/to/release-evidence.json. Exit 1 means blocked, exit 2 means malformed evidence, and exit 0 means only that the complete submitted checklist can proceed to independent artifact review. The checker has no release or board-writing side effects.

A 4–6 week cycle is a workstream sequence, not an unconditional shipping promise. Missing binary smokes, coverage, repeated DRC, corpus evidence, local reconciliation or independent review means no full v0.13 hybrid-router release claim. Do not weaken thresholds to meet a date.

Follow-on program

v0.14 targets broader conversion, blind/buried/microvia policy, side-aware placement and atomic multi-operation apply design. v0.15 targets calibrated placement quality and larger held-out co-optimization. v0.16+ targets actual authoritative SI/PI, thermal, EMC and fabrication backends, calibration, repeated evidence and human/physics validation. Until those gates close, describe CopperMCP as AI-extensible, evidence-first design automation, not a certified autonomous electrical-engineering agent.