Skip to content

fix: read metadata and runtime version at the head, not the finalized block - #154

Open
n13 wants to merge 2 commits into
mainfrom
n13/client-metadata-at-head
Open

fix: read metadata and runtime version at the head, not the finalized block#154
n13 wants to merge 2 commits into
mainfrom
n13/client-metadata-at-head

Conversation

@n13

@n13 n13 commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Found live: after Heisenberg enacted spec 148, quantus tech-referenda config still showed only track 0 with the 144 decision deposit of 1000 UNIT, so it looked like the fast_upgrade track had not shipped. It had. The CLI was reading the pre-upgrade runtime.

Cause

OnlineClient::from_rpc_client fetches metadata at latest_finalized_block_ref(). QPoW finality trails the head by ~100 blocks, so for roughly 20 minutes after any runtime upgrade enacts the client decodes against the old runtime.

Measured on Heisenberg at the time:

apply landed in block: 977079
finalized block:       977001   (78 blocks / ~15 min behind)

metadata @ head       -> fast_upgrade PRESENT   (spec 148)
metadata @ finalized  -> fast_upgrade ABSENT    (spec 144)

The client was also inconsistent with itself: subxt reads the runtime version with state_getRuntimeVersion and no block argument, which answers at the head. So the version said 148/6 while the metadata was 144. Signing was not affected by this (the version was already right); the earlier draft of this PR claimed it was, and that was wrong.

Consequences

  1. Governance config reads stale, the symptom above.
  2. Calls and storage added by the upgrade look absent, so commands that need them fail for ~20 minutes after an upgrade that in fact succeeded.
  3. Extrinsics encode against the old metadata. A call whose index or arguments changed in the upgrade is encoded wrong.

Change

QuantusClient::connect no longer uses subxt's constructor. It takes one best-block hash and reads the runtime version and the metadata at that hash, then builds the client with from_backend_with. The runtime identity gate validates the same state_getRuntimeVersion response, so there is no second, unpinned request that could disagree. Metadata is negotiated the way subxt does it (v16, v15, v14, then legacy), so the client keeps the same metadata version it had before rather than dropping to the v14 state_getMetadata returns.

Historical reads keep the runtime that produced the block. QuantusClient::at_block(hash) returns a client whose metadata and runtime version come from that block. It reuses the connection, and when the block runs the same runtime as the head it returns the existing client without fetching metadata again (spec version cannot repeat across upgrades, so that comparison is exact). Routed through it:

  • quantus events --block / --block-hash / --finalized
  • quantus block analyze
  • quantus storage --block
  • the SDK's at_finalized_block, and the wormhole transfer-count reads at that block now use the block's own client
  • the upgrade exercise scenario, which scans events across the upgrade boundary by design

Reads at a transaction's inclusion block are left on the head client: the transaction was just built and signed against that runtime. block list is also left alone; it reads only Timestamp::Now and System::EventCount, and pinning per block would add one RPC to each of up to 10k blocks.

A failure to read the version or metadata is an error, not a fallback. Skipped metadata versions during negotiation are logged in verbose mode.

Tests

A mock node (jsonrpsee server, dev-dependency only) serves spec 148 at the head and spec 144 at the finalized block, and records which block every metadata request named:

  • connect installs the head runtime version and requests metadata at the head hash, not the finalized one
  • at_block(finalized) yields a client on 144/3 with metadata requested at that hash, leaves the head client untouched, and at_block(head) fetches nothing new

Plus a unit test for runtime-version JSON parsing.

Verification

Same command that exposed it, against live Heisenberg on 148:

• Track #0: tech_collective_members   Decision Deposit: 10000000000000   (10 UNIT, was showing 1000)
• Track #1: fast_upgrade              Prepare 50 blocks (~10min)
                                      Confirm 50 blocks (~10min)
                                      Min Enactment 50 blocks (~10min)

Track #1 now appears and track #0 shows 148's deposit. quantus events --finalized against a finalized block still on 144 decodes every event.

  • SKIP_CIRCUIT_BUILD=1 cargo test --release --lib
  • SKIP_CIRCUIT_BUILD=1 cargo clippy --all-targets --locked -- -D warnings
  • cargo +nightly-2026-08-31 fmt --all -- --check

… block

subxt pins a client's metadata and runtime version to the latest finalized block.
QPoW finality trails the head by ~100 blocks, so for roughly 20 minutes after a
runtime upgrade enacts the CLI keeps talking to the old runtime:

- governance config reads stale. After Heisenberg enacted spec 148,
  tech-referenda config still listed only track 0 with the 144 decision deposit
  of 1000 UNIT, so the fast_upgrade track looked like it had not shipped.
- calls and storage added by the upgrade appear absent.
- transaction_version is wrong, which signs extrinsics the chain rejects. That
  matters here because 144 -> 148 moves it from 3 to 6.

Re-point both at the head after connecting, matching every other read path in the
CLI (#152 did the same for collect-rewards proofs). A failure to read them is an
error rather than a fallback, since silently continuing would leave the client on
finalized metadata -- the bug this fixes.
@n13 n13 added the bot-review Request automated review from review-bot label Sep 3, 2026

@n13 n13 left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewer model: GPT Sol

Verdict (advisory): Request changes

Blocking findings:

  1. src/chain/client.rs:149 installs head metadata into the one shared OnlineClient, but not every consumer reads the head. events --finalized, events --block, events --block-hash, block analysis, and SDK finalized paths pass older block hashes to Subxt, which still decodes them with this single head metadata snapshot. This is a concrete regression across an upgrade boundary. During review, Heisenberg's head was spec/tx 148/6 while finalized block 977072 was still 144/3 and the metadata payloads differed. Before this patch, events --finalized decoded all 10 events in block 977072. The exact PR binary, using spec-148 head metadata, fails on that same block after four events with Could not decode Phase, variant doesn't exist. Please keep finalized/historical reads on metadata from their target block (or separate the head-oriented client from block-specific clients) instead of globally assuming every caller targets the head.

  2. src/chain/client.rs:187-205 does not capture one coherent head snapshot. state_getMetadata and state_getRuntimeVersion each resolve an omitted block argument independently, so an upgrade block becoming best between the two calls leaves old metadata paired with a new runtime version. The identity gate at lines 155-167 then makes a third unpinned version request and cannot detect that mismatch. Because this client does not run Subxt's runtime updater, the inconsistent pair persists for the command and can reproduce the missing-call or invalid-extrinsic behavior this patch is intended to eliminate. Capture the best block hash once, request both metadata and runtime version at that hash, validate that same version response, and add coverage for the upgrade-boundary case.

Non-blocking accuracy note: Subxt 0.44.3's LegacyBackend::current_runtime_version already calls state_getRuntimeVersion(None) at the head; its metadata fetch is the part pinned to the finalized hash. The comments and PR rationale should not claim both values were previously finalized.

Validation on exact head 250ef8a0138acdd5fc129b266f9867beec2b001f:

  • git diff --check - passed.
  • taplo format --check --config taplo.toml - passed.
  • cargo +nightly-2026-08-31 fmt --all -- --check - passed.
  • cargo metadata --locked --no-deps --format-version 1 - passed.
  • SKIP_CIRCUIT_BUILD=1 cargo test --release --locked --lib chain::client::tests - 3 passed; the new retargeting path itself has no automated coverage.
  • SKIP_CIRCUIT_BUILD=1 cargo clippy --all-targets --locked -- -D warnings - passed.
  • Live Heisenberg tech-referenda config - passed and showed both spec-148 tracks, confirming the intended head-read fix.
  • Live Heisenberg historical-event reproduction at block 977072 (0x705dadae...c685f5) - failed only after the client was retargeted to spec-148 metadata, as described above.

All current hosted checks pass, including Ubuntu/macOS builds and tests, examples, strict analysis/docs, formatting, security audit, and dependency cooldown.

@n13 n13 removed the bot-review Request automated review from review-bot label Sep 3, 2026
…heir own runtime

Review of #154 found two problems with retargeting the shared client at the head.

Every historical read decoded with head metadata. After an upgrade, `events
--finalized`, `events --block`, `block analyze` and the SDK's finalized path
failed on blocks the old runtime produced ("Could not decode Phase"). Add
`QuantusClient::at_block(hash)`: a client whose metadata and runtime version
come from that block. Same runtime as the head reuses the existing client, so
the common case costs one `state_getRuntimeVersion`. Route the historical
readers through it.

The head snapshot was not coherent. `state_getMetadata` and
`state_getRuntimeVersion` each resolved the head on their own, so an upgrade
landing between them paired old metadata with the new version, and the
identity gate made a third unpinned request. Take one best-block hash, read
version and metadata at that hash, validate the same version response, and
build the client with `from_backend_with`. Metadata is negotiated the way
subxt does it (v16 first), instead of dropping to v14 via `state_getMetadata`.

subxt already read the runtime version at the head; only metadata was pinned
to the finalized block. The earlier comment claiming both were finalized was
wrong.

Tests: a mock node serving spec 148 at the head and 144 at the finalized block
checks that connect names the head hash for both reads, and that `at_block`
pins a pre-upgrade block without touching the head client or refetching for a
same-runtime block. jsonrpsee's server feature is a dev-dependency only.
@n13 n13 added the bot-review Request automated review from review-bot label Sep 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bot-review Request automated review from review-bot

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant