feat(modelkit): add swin2sr support and canonical cpu recipe

[fangyangci]

Summary

Adds canonical CPU fp32/fp16 recipes for caidas/swin2SR-classical-sr-x2-64 image-to-image super-resolution and fixes shared compare-mode task validation so the model can reach tensor comparison without adding a broad image-to-image benchmark evaluator. The shipped Effort/Outcome is L2, and the highest Goal verdict is L2 PASS with full planned tuple coverage and no deferred tuples. The fp16 artifact preserves numeric parity but is slower than fp32 on the tested CPU host.

Model metadata

What the model does

Upscales an RGB image by 2x using a Swin Transformer V2 restoration backbone and pixel-shuffle reconstruction head.

• Evidence/confidence: pinned checkpoint caidas/swin2SR-classical-sr-x2-64@cee1c923c6a37361c6e5650b65dcf4be821e5d52, its model card/config, and Swin2SR paper arXiv:2209.11345v1 (verified).

Primary user stories

• Upscale a low-resolution image for display or downstream inspection. Evidence: model card and checkpoint pipeline_tag=image-to-image (verified).
• Restore detail in classical super-resolution inputs without changing the application task contract. Evidence: model card and Swin2SR paper (verified).
• Deploy the checkpoint as a CPU ONNX model and compare fp32 and fp16 latency/correctness. Evidence: PR feat(modelkit): add swin2sr support and canonical cpu recipe #1204 scope and current-main inspect/config/build probes (mapped).

Supported tasks

• image-to-image across the checkpoint, Transformers, Optimum ONNX, and WinML surfaces. Evidence: checkpoint pipeline_tag, Swin2SRForImageSuperResolution, and the current-main WinML inspect/recipe-free build (verified).

Model architecture

Swin2SRForImageSuperResolution
|-- shallow feature extraction (input convolution)
|-- Swin2SR encoder/backbone
|   |-- patch embedding (patch size 1, embedding width 180)
|   `-- residual Swin Transformer groups x6
|       `-- Swin Transformer V2 blocks x6 per group
`-- pixel-shuffle reconstruction head (2x RGB output)

• Source/confidence: pinned checkpoint config, Transformers Swin2SR source, and current-main hierarchy trace (verified).

Validation and support evidence

1. Baseline

• Pinned current main: 24cccfb23d6e3c3b93ae9bb94399c326cee8ecea; WinML 0.3.0.
• Recipe-free CPU build: PASS in 32.1 s. Contract: pixel_values float32 [1,3,64,64] to reconstruction float32 [1,3,128,128].
• Baseline fp32 CPU perf: PASS, 100 iterations after 10 warmups; mean 772.867 ms, p50 770.268 ms, throughput 1.29 samples/s, total RSS delta 196.06 MB.
• Starting auto-config: task image-to-image, AutoModelForImageToImage, opset 17, fixed float32 [1,3,64,64] input, eager Transformers attention, hierarchy tags, GELU/highdimRTR-lowdimRTR/MatMul-Add optimizations, no quantization, and no compilation.
• Baseline Goal floor: L1 because build and perf passed. Compare Eval failed before inference: _resolve_task rejected image-to-image against the metric evaluator registry even though compare mode dispatches compare-tensor; the real Set5 decode and preprocessing had passed.
• Refresh decision: FULL-RERUN. Although the baseline SHA remained current and its ancestor check passed, the old candidate was not patch-equivalent (89 main-only / 2 PR-only commits, five changed paths), lacked fp32 coverage and the compare-mode validator fix, and had no acceptance-eligible exact-lock execution. The frozen model identity/profile/source citations were reused; config, build, perf, parity, Eval, Analyze, and quality gates were rerun on the reconstructed final head.
• Optimum probe: vendor and WinML both expose feature-extraction and image-to-image; WinML adds no task registration (VENDOR-ONLY). A direct canonical Optimum model-config import fails upstream because _CAN_RECORD_REGISTRY is unavailable under the repository lock, but supported WinML inspect/config/build paths resolve Swin2srOnnxConfig through TasksManager and are not blocked.

2. Goal

• Effort: L2.
• Goal ceiling: L2.
• Outcome: L2.
• Required tuples: CPU/cpu fp32 and fp16. L0 requires successful recipe builds, L1 requires measured runtime perf for each tuple, and L2 requires finite, shape-compatible numeric comparison against the PyTorch reference. Charter revision 2 reissued the scope after the two prior blockers were resolved and exact-lock current-main execution became available.

3. Outcome

• Shipped tier: L2; highest Goal verdict: L2 PASS; coverage: full; deferred tuples: none.
• Recipes:
  • examples/recipes/caidas_swin2SR-classical-sr-x2-64/cpu/cpu/image-to-image_fp32_config.json
  • examples/recipes/caidas_swin2SR-classical-sr-x2-64/cpu/cpu/image-to-image_fp16_config.json
• Shared code/tests:
  • src/winml/modelkit/eval/evaluate.py
  • tests/unit/eval/test_eval.py
  • tests/unit/models/test_swin2sr_support.py
• README status: examples/recipes/README.md is untouched.
• Model findings captured swin2sr-001 through swin2sr-011: fixed fp32/fp16 IO contract; hierarchy facts; mode-driven compare dispatch; exact perf and parity behavior; fp16 initializer/size change; component mapping gaps; and bounded static-rule findings.
• Methodology: no friction was found in this run and no new methodology finding was warranted. Existing _meta-109 was reused for explicit provider/device static Analyze with --no-run-unknown-op --no-check-optim; it was not duplicated.
• Lane A remains separate in Draft PR gim-home/ModelKitArtifacts#222, commit fc63c516a5eb68aee6b6df15ac2d7c9585a05646, containing only copilot-skills/dev_skill/adding-model-support/model_knowledge/swin2sr.json.

4. Per-EP/device/precision results and Functional smoke Eval

Tier	EP / Device	Precision	Verdict	Mean	p50	Throughput	RAM delta	Local VRAM delta	Shared VRAM delta
L0	CPUExecutionProvider / cpu	fp32	PASS	-	-	-	-	-	-
L0	CPUExecutionProvider / cpu	fp16	PASS	-	-	-	-	-	-
L1	CPUExecutionProvider / cpu	fp32	PASS	776.928 ms	774.658 ms	1.29 samples/s	+196.0 MB	+0.0 MB	+0.0 MB
L1	CPUExecutionProvider / cpu	fp16	PASS	942.144 ms	936.591 ms	1.06 samples/s	+243.1 MB	+0.0 MB	+0.0 MB

Both L0 artifacts are self-contained opset-17 ONNX models with float32 [1,3,64,64] input and float32 [1,3,128,128] output. fp32 has 2,096 nodes, 695 initializers (672 FLOAT, 23 INT64), and 70,912,129 bytes. fp16 has 2,098 nodes, 695 initializers (672 FLOAT16, 23 INT64), and 35,854,351 bytes. The tested fp16 CPU path is slower than fp32: mean/p50 increase from 776.928/774.658 ms to 942.144/936.591 ms, throughput falls from 1.29 to 1.06 samples/s, and RSS delta rises from 196.0 to 243.1 MB.

L2 numeric comparison:

Precision	Source	Cosine	Max abs	MSE	PSNR	SQNR	Shapes	Finite	Verdict
fp32	Sole functional-smoke compare invocation	0.9999999999999974	2.980232238769531e-07	3.895976604118463e-15	144.10372975587683 dB	141.8807465957932 dB	candidate/reference [1,3,128,128]	true	PASS
fp16	Tester compare-tensor harness; not an Eval invocation	0.9999999818059054	0.00046575069427490234	2.1871883008097977e-08	76.61103141598643 dB	74.3880482559028 dB	candidate/reference [1,3,128,128]	true	PASS

Functional smoke Eval: exactly one final-head FP32 CPU compare Eval was run. No fp16 Eval is claimed.

• Candidate: 69847a89be6f7085d003ba08b003eb182cc5f88a; CPUExecutionProvider / cpu / fp32.
• Real pinned data: eugenesiow/Set5@d8b579a20afde95b4d8ed6bf6383447d33027295, config data/Set5_LR_x2.tar.gz, member Set5_LR_x2/baby.png.
• Decode/preprocess: PNG, RGB, 256x256; deterministic crop [100,100,156,156]; pixel_values float32 [1,3,64,64], observed range [0.0313725508749485,1.0].
• Requested samples: 1; processed samples: 1.
• Fan-out caps: dataset rows 1; frames/crops 1; candidate outputs 1; reference outputs 1; candidate labels/prompts none; beams none; sequence/generation length none.
• Schema, label semantics, and prediction semantics: verified. Output reconstruction is [1,3,128,128].
• Raw metrics: cosine 0.9999999999999974; max abs 2.980232238769531e-07; MSE 3.895976604118463e-15; PSNR 144.10372975587683 dB; SQNR 141.8807465957932 dB; finite values true.
• This is an operability-only functional smoke proving the newly reachable end-to-end compare path. It is not representative accuracy or benchmark-quality evidence.

5. Delta

Recipe delta from the current-main auto-config:

Recipe	Comparison	Exact structured delta
image-to-image_fp32_config.json	IDENTICAL	None after _note stripping
image-to-image_fp16_config.json	CHANGED	/quant: null -> mode=fp16, samples=10, task=image-to-image, model_id=caidas/swin2SR-classical-sr-x2-64, model_type=swin2sr, fp16_keep_io_types=true

The fp16 /quant change declares the required precision while preserving the baseline model, loader, export IO, attention compatibility, optimization, and compile semantics. Both recipes load through WinMLBuildConfig; the fp32 structure resolves to fp32 and fp16 quant.mode resolves to fp16. The change is reducible consistently with the charter, and examples/recipes/README.md remains untouched.

Code delta: _resolve_task in src/winml/modelkit/eval/evaluate.py remains model-task-driven, but skips ordinary metric-registry validation when mode=compare because evaluator dispatch selects the task-agnostic compare-tensor evaluator. Tests cover the shared resolver/dispatcher/evaluate path and Swin2SR vendor registration/default task without an unused plain winml import.

Bug fix explanation:

1. Symptom and minimal trigger: winml eval --mode compare with a resolvable image-to-image model and real input failed before inference with "Task 'image-to-image' is not supported," even though compare mode is tensor comparison.
2. Root cause: _resolve_task validated the resolved model task against the labeled-dataset metric evaluator registry before get_evaluator_class could remap compare mode to compare-tensor. Therefore a valid HF model task with no metric evaluator was rejected on the wrong registry boundary.
3. Changed symbol and mechanism: _resolve_task now preserves the resolved model task for HF reference loading and bypasses metric-registry validation only for config.mode == 'compare'; evaluator dispatch still selects TensorSimilarityEvaluator through compare-tensor.
4. General rule: the decision is mode-driven and model-ID-free. It applies to any resolvable model task using compare mode and contains no Swin2SR/checkpoint hardcoding.
5. Compatibility and blast radius: ordinary mode=onnx still rejects image-to-image; existing supported tasks resolve in both modes; compare retains the model task and tensor metrics; two-ONNX compare remains task/model-id optional; recipe-owned semantics are not overridden. The intentional change is that compare mode accepts a resolvable task without a labeled-dataset metric evaluator.
6. Regression evidence: the focused resolver/dispatcher/evaluate and Swin2SR tests passed 26/26. Full affected partitions passed: Analyze 1,526 passed / 45 skipped; models/loader/datasets/export 1,526 passed / 6 skipped / 2 xfailed; optim 714 passed / 16 skipped / 1 xfailed; commands/config/build/compiler/session/eval 3,611 passed / 9 skipped / 1 warning; remaining unit/regression/CLI 870 passed / 2 skipped / 1 deselected / 1 warning. Ruff, mypy over 435 source files, license insertion, and diff checks passed.

6. Analyze summary - component level and op level

Both artifacts returned ANALYZE-PARTIAL-SUCCESS (exit meaning: static analysis contains partial/unknown classifications). These are bounded static rule classifications only; they do not establish runtime execution, support, or fallback behavior.

Component-level summary

Artifact	Architecture coverage	Mapping	Actionable EP findings
fp32	shallow/global features; patch embeddings; 6 SwinV2 stages; residual body; 2x pixel-shuffle reconstruction	1,517 mapped; 579 optimized nodes unmapped; partial confidence	None
fp16	shallow/global features; patch embeddings; 6 SwinV2 stages; residual body; 2x pixel-shuffle reconstruction	1,517 mapped; 581 optimized nodes unmapped; partial confidence	QNN GPU and QNN NPU: all 72 attention Expand nodes partial; none unsupported

The mapping gap is retained: final optimization may remove or rewrite export-scope names, leaving 579 fp32 and 581 fp16 optimized scaffolding nodes without a semantic scope.

Op-level summary

Artifact	Graph	Dominant operator counts	EP roll-up
fp32	2,096 operators / 20 types	Reshape 572; Transpose 338; Gemm 180; Add 152; Slice 146; MatMul 108; LayerNormalization 74; ReduceL2 72	NvTensorRTRTX GPU, OpenVINO CPU/GPU/NPU, and QNN GPU/NPU have no partial or unsupported static classifications
fp16	2,098 operators / 21 types	Reshape 572; Transpose 338; Gemm 180; Add 152; Slice 146; MatMul 108; LayerNormalization 74; ReduceL2 72	NvTensorRTRTX GPU and OpenVINO CPU/GPU/NPU have no partial or unsupported static classifications; QNN GPU/NPU classify Expand partial (72 nodes), none unsupported

CPU/cpu and VitisAI/NPU are all-unknown rule-coverage gaps for both artifacts. No runtime support inference is made. This bounded explicit-provider scan follows existing methodology finding _meta-109.

7. Reproduce commands

$OUT='temp/swin2sr-support-repro'
winml build -c examples/recipes/caidas_swin2SR-classical-sr-x2-64/cpu/cpu/image-to-image_fp32_config.json -m caidas/swin2SR-classical-sr-x2-64 -o $OUT/fp32
winml build -c examples/recipes/caidas_swin2SR-classical-sr-x2-64/cpu/cpu/image-to-image_fp16_config.json -m caidas/swin2SR-classical-sr-x2-64 -o $OUT/fp16 --precision fp16
winml perf -m $OUT/fp32/model.onnx --ep cpu --device cpu --memory
winml perf -m $OUT/fp16/model.onnx --ep cpu --device cpu --memory
$env:WINMLCLI_RULES_DIR='<rules-root>'
winml analyze -m $OUT/fp32/model.onnx --ep openvino --device cpu --no-run-unknown-op --no-check-optim
winml eval --mode compare -m $OUT/fp32/model.onnx --model-id caidas/swin2SR-classical-sr-x2-64 --task image-to-image --input-data <set5-baby-input.npz> --skip-build --device cpu --ep cpu

[ssss141414]

APPROVE

Independent post-PR review completed for exact head 69847a89be6f7085d003ba08b003eb182cc5f88a against base/current main 24cccfb23d6e3c3b93ae9bb94399c326cee8ecea from a fresh detached checkout.

Scope and engineering review

• The diff is exactly five files: paired CPU fp32/fp16 recipes, the shared _resolve_task change, and focused eval/Swin2SR tests. examples/recipes/README.md is unchanged and git diff --check is clean.
• The implementation is general and mode-driven: compare mode preserves any resolvable model task and dispatches TensorSimilarityEvaluator; ordinary ONNX Eval still rejects tasks without metric evaluators; supported tasks are unchanged; taskless two-ONNX compare still bypasses HF task requirements. No model ID/type hardcoding was added.
• The tests exercise each invariant above and the prior unused plain winml import is absent. The Swin2SR tests verify vendor registration/default task rather than restoring a stale architecture shim.

Independent validation

• Tester seals verified before reuse: execution 115/115 and consolidated verdict 25/25 hashes match.
• Both recipes build with explicit completion banners. fp16 used --precision fp16 without --no-quant, has 672 FLOAT16 initializers, and is 35,854,351 bytes versus fp32's 70,912,129 bytes.
• Independent non-Eval CPU inference reproduced exact named I/O and finite outputs for both artifacts; fp16-vs-fp32 cosine was 0.9999999818062772 with max abs 0.00046563148498535156. This preserved the tester's exactly-one raw Eval invariant.
• L1: fp32 mean/p50 776.928/774.658 ms, 1.29 samples/s, +196.0 MB RSS; fp16 942.144/936.591 ms, 1.06 samples/s, +243.1 MB RSS.
• L2: fp32 cosine 0.9999999999999974; fp16 harness cosine 0.9999999818059054; shapes/finite/metric checks pass. Exactly one final-head fp32 CPU Set5 functional smoke processed one pinned real row with explicit fan-out caps and no benchmark claim.
• Raw HTP/build/analyze structures agree with the report: 12,091,571 parameters, 884 modules/278 traced, 16 explicit Analyze rows, documented 579/581 mapping gaps, and 72 fp16 QNN partial Expand nodes with no runtime-support inference.

Quality and shipment gates

• uv run ruff check src/ tests/: PASS. uv run mypy -p winml.modelkit: PASS, 435 source files.
• Focused: 26 passed. All five non-hardware CI partitions: 8,247 passed total (Analyze 1,526; Models 1,526; Optim 714; Commands 3,611; Remaining 870), with only expected skips/xfails/deselections and two non-failing warnings.
• PR body hierarchy/value fidelity, FULL-RERUN baseline, E/G/O L2, two perf rows, fp32/fp16 parity, one Set5 smoke, Analyze summaries/gaps, public commands, no local paths, and README status all verified.
• Discussion fully paginated: 0 conversation comments, 1 line comment, 1 submitted review, 1 review thread; the former unused-import thread is resolved/outdated. Unresolved threads: 0.
• Lane A gim-home/ModelKitArtifacts#222 is Draft at exact commit fc63c516a5eb68aee6b6df15ac2d7c9585a05646, has label model-scale-by-skill, one-file scope, one green check, and zero unresolved discussion.
• Final Lane B state immediately before this comment: unchanged head, Draft, label model-scale-by-skill, 9/9 checks COMPLETED/SUCCESS.

Engineering observations: fp16 is slower on the tested CPU; CPU/VitisAI static-rule coverage and optimized-node mapping gaps remain explicitly reported without support inference. These are non-blocking. Owner routes: none.