engine_newPayload strict-fields rule (null = not provided) unenforced for transactions and remaining payload fields

[flcl42]

Cancun engine_newPayloadV3 §1, inherited by V4/V5/V6: "MUST check that provided set of parameters and their fields strictly matches the expected one and return -32602 … Any field having null value MUST be considered as not provided." #13714 covers withdrawals/blobGasUsed/excessBlobGas. The rule is still unenforced for the rest of the payload surface:

• transactions is not null-checked anywhere on the V3+ path (ExecutionPayloadParams.ValidateEngineApiVersionParams checks withdrawals, blockAccessList, slotNumber and blobVersionedHashes presence only). A null/missing transactions falls through to TryGetTransactions/TryGetBlock and surfaces as INVALID or an internal error instead of -32602.
• Base ExecutionPayload scalar fields (parentHash, feeRecipient, stateRoot, receiptsRoot, logsBloom, prevRandao, blockNumber, gasLimit, gasUsed, timestamp, extraData, baseFeePerGas, blockHash) carry no [JsonRequired] and non-nullable defaults (= Keccak.Zero, = [], …), so a missing key deserializes to the default and fails later as INVALID (hash mismatch) instead of -32602.
• [JsonRequired] present-but-null semantics (V3 blobGasUsed/excessBlobGas, V4 blockAccessList/slotNumber): verify System.Text.Json rejects explicit JSON null the same as a missing key for each — the STJ required-check is the only gate for those fields and its null treatment must match "counts as not provided".

Suggest one pass over every field of ExecutionPayload V1–V4 shapes (plus the sidecar params) against the rule, with per-field error-code tests. Related: #13714 (the three named fields), #13712 (fork windows).

Spec: cancun.md – engine_newPayloadV3 §1.

Found in the Glamsterdam EIP-7732 ePBS EL-surface audit (sec4 scan).

[arunimshukla]

I validated the remaining cases against the regression-only baseline for #13714: https://github.com/arunimshukla/nethermind/actions/runs/35901279330

The baseline distinguishes omission from explicit null:

• omitted blob fields already fail deserialisation/validation;
• explicit null values pass through to the handler, which is the path fixed by fix(engine): reject null required Cancun payload fields #13734;
• omitted withdrawals also reached the handler and was covered by fix(engine): reject null required Cancun payload fields #13734.

For the other fields, transactions: null should follow the same strict-fields rule and be rejected as -32602 before block reconstruction. The focused fix appears to be one null check in the V3+ parameter validation plus a transactions value in the existing regression test.

I also checked the fork-specific gates called out here: explicit null blockAccessList and slotNumber are already rejected by the existing BlockLevelAccessListsEnabled and IsEip7843Enabled checks, returning -32602. I would leave the base ExecutionPayload required-field question separate because adding [JsonRequired] affects V1/V2 and the Optimism/Taiko payload shapes. @flcl42, would you prefer that broader change as one cross-version patch or separate per-payload changes? I will wait for maintainer direction before touching it.

[flcl42]

Thanks @arunimshukla, and thanks for the #13714 fix (it landed as #13747).

I tested current master (5542f1c7b0) through JsonRpcService on engine_newPayloadV3 and engine_newPayloadV5. For every payload field I sent explicit null and also left the key out. Results:

• transactions: null already returns -32602. The Transactions setter throws ArgumentNullException while the params are being bound, so a null check in ValidateEngineApiVersionParams would never run. The real gap is a missing transactions key: it falls back to [] and goes on to block validation. That gives INVALID, or even VALID for an empty block whose hash matches.
• Missing base fields (parentHash … blockHash) all fall back to their defaults and return INVALID instead of -32602.
• Explicit null on base fields:
  • blockNumber, gasLimit, gasUsed, timestamp and baseFeePerGas return -32602.
  • parentHash, feeRecipient, stateRoot, receiptsRoot, logsBloom and prevRandao return INVALID.
  • blockHash: null returns -32603 (a NullReferenceException) on both V3 and V5, and extraData: null returns -32603 on V3.
• I agree on the third bullet. After fix(engine): reject null required Cancun payload fields #13747, blobGasUsed, excessBlobGas, blockAccessList and slotNumber all return -32602, whether sent as null or left out. That bullet is resolved.

On your question: please do it as one patch, limited to the V3+ engine_newPayload path.

• Every ExecutionPayloadV3/V4 field must be present and non-null, otherwise return -32602.
• The same change should cover the -32603 cases for null blockHash/extraData.
• Please add one parameterized test over field × {null, omitted} × version, in the style of NewPayload_should_reject_null_or_missing_required_fields.

Please leave V1/V2 and TaikoExecutionPayload as they are. TaikoExecutionPayload deliberately turns a null or missing transactions into [], so adding [JsonRequired] to the base ExecutionPayload would break it.

The defaults (= Keccak.Zero, = [], …) make a missing key look like a real value once deserialization is done. So presence has to be recorded while the params are being bound; it can't be checked afterwards.

The Optimism and Taiko V3 payloads derive from ExecutionPayloadV3, so please check that their behaviour and tests stay as intended.

🤖 AI agent (OpenCode / Claude Opus 5.5) on behalf of @flcl42