Skip to content

Publish portable workflow and value interchange schemas - #256

Draft
sunapi386 wants to merge 3 commits into
mainfrom
work/206-interchange-schema
Draft

Publish portable workflow and value interchange schemas#256
sunapi386 wants to merge 3 commits into
mainfrom
work/206-interchange-schema

Conversation

@sunapi386

@sunapi386 sunapi386 commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Publish the portable graph interchange contract in schema/: five generated, standalone Draft 2020-12 documents cover workflows, engine value-type metadata, registered Value instances, tagged Result instances, and erasable hints. Include runnable nested ForEach and Result workflow examples and document exporter obligations to resolve and inline host workflow references, pin versions, preserve provenance without making execution depend on it, and reject unresolved dependencies.

The design distinguishes metadata from instance validation: Result's existing ok/err metadata remains unchanged, while the instance schema uses tagged oneOf branches with required payloads. Generate recursive Workflow/ValueSchema definitions together and share them instead of duplicating the entire family in each node parameter contract. Current built-in parameter contracts are selected by node name/version; extension names remain open and require receiver validation. The runtime loader and NodeTypeInfo.parameter_schema API are unchanged.

The portable export profile requires resolved node versions and complete type metadata. Documentation distinguishes structural validation from runtime type/resource checks, including the existing missing-required-input-edge limitation (#95). Source-export discovery makes regeneration independent of installed entry-point metadata. Add JSON Schema validation as a development dependency and a CI generation check. Runnable examples regenerate from current node versions and parameter defaults alongside the schemas.

Validation: 1,057 tests passed, 1 existing expected failure; Ruff lint/format, Pyright, interchange regeneration checks passed. The 42 new contract cases include offline reference closure, Draft 2020-12 metaschema validation, stale-artifact rejection, malformed tags/parameters/hints, an unavailable extension, and round-trip execution under both executors. Pydantic emits its existing warning that a class-valued default (Value) is not JSON serializable and is excluded from schema defaults; generation and all checks still succeed.

Depends on #251 for explicit Value identities and #261 for record-field defaults. The combined integration branch also includes #259, which implements required-edge validation and updates the corresponding documentation. Integration with new public node types requires rerunning uv run python scripts/generate_interchange_schema.py.

Closes #206

@sunapi386
sunapi386 deleted the branch main September 10, 2026 14:22
@sunapi386 sunapi386 closed this Sep 10, 2026
@sunapi386 sunapi386 reopened this Sep 10, 2026
@sunapi386
sunapi386 changed the base branch from work/224-schema-identity to main September 10, 2026 14:26
@sunapi386

Copy link
Copy Markdown
Contributor Author

Holding this, on the gate rather than on the code.

#206 is still labelled needs-decision, and the decision it is waiting on is exactly what this PR performs: whether to publish a portable interchange schema from this repo at all. Publishing 10,000 lines of generated JSON Schema into a public repo is a public API surface commitment, and it is the kind of commitment that is much cheaper to not make than to withdraw. A design produced without that decision having been taken does not settle it.

To be clear about what is not the problem here, because the shape of this PR is better than its size suggests:

  • The 11,000 lines are generated artifacts, not hand-written. scripts/generate_interchange_schema.py produces them.
  • There is a freshness gate. tests/test_interchange_schema.py plus the CI step means the committed JSON cannot silently drift from the code, which is the same defect class as Generic Value types are absent from the typecast graph entirely #246 and the right way to have solved it.
  • The examples (schema/examples/foreach-workflow.json, result-workflow.json) are the correct way to make a spec checkable rather than aspirational.

So this is not a rewrite. It is a PR waiting on a decision that has an owner, and the owner is not the reviewer.

Two questions worth answering when that decision is made, both of which change the artifact:

  1. Does publishing here rather than in a separate spec repo still hold, given how much surface this turned out to be? The issue's reasoning was that a separate repo "creates a second seam to keep honest before anything needs one", which is right, but the size of what is being published is new information.
  2. What stability does publishing imply? The engine is at a release candidate. A published schema that changes shape between release candidates is worse than no published schema, because consumers will pin to it. The erasability rule the issue calls out, that a host-specific node must be erasable at export into a pure public one, is the part with real consumers behind it and could ship ahead of the full schema dump.

Leaving as a draft with the branch intact. Rebase onto main when it is unblocked; it currently carries the merged #251 work as its base.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Publish the interchange schema from schema/

1 participant