Skip to content

feat(api): accept a client-supplied idempotency key on the eight non-convergent writes #241

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
LKSNDRTMLKV wants to merge 2 commits into
base: main
Choose a base branch
from
Open

feat(api): accept a client-supplied idempotency key on the eight non-convergent writes #241

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
238a064
feat(api): accept a client-supplied idempotency key on the eight non-…
LKSNDRTMLKV Sep 4, 2026
258e8e8
fix(ci): scope the append-only migration gate to the sql files it pro…
LKSNDRTMLKV Sep 4, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 49 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,55 @@ under the pre-1.0 conventions in [VERSIONING.md](docs/governance/VERSIONING.md):

### Added

- **Eight write endpoints now accept a client-supplied `Idempotency-Key`.**
Send the same key with the same body and the first outcome is returned rather
than a second resource being created. The replay carries
`Idempotency-Replayed: true`, so a client can tell a retry that worked from
one that duplicated without comparing anything.

Which routes get a key is decided by **effect, not verb**: the test is whether
a replay creates a second thing, or spends something that cannot be un-spent.
That is `POST /dpp`, evidence generation, plugin install, and the five creates
behind API keys, webhooks, facilities, operator identifiers and bulk import.
`PUT` and every lifecycle transition are deliberately excluded — they converge
on their own — and a key sent to one of them is a `400`, not a silent no-op.
Accepting the header where nothing records it would advertise a protection
that is not there.

The key is bound to a SHA-256 of the **raw** request body. Not of a
canonicalised form: canonicalising would invent a normalisation this API does
not otherwise have, and a client that re-serialises with different member
order has changed its request. Reusing a key with a different body is
refused with `422` under its own problem type
(`.../idempotency-key-reuse`) — built directly rather than through the shared
helper, which derives the type URI from the status reason and would have made
it indistinguishable from every other validation failure.

Keys are scoped to the authenticated caller and the matched route template,
honoured for 24 hours, and swept hourly. A concurrent duplicate gets `409`
with `Retry-After`; a claim orphaned by a crash is reclaimed after 60 seconds.
If the key store is unreachable the write is **refused**, not run: executing
with no record produces exactly the outcome the caller asked to be protected
from.

Two carve-outs, both deliberate and both documented on the routes themselves.
`POST /api-keys` and `POST /webhooks` return a secret once, so the secret is
never stored — their replay returns the created resource with
`"secretAlreadyDelivered": true` in place of it. That is the only shape
divergence in the set; the alternative was parking a live credential in a
table for a day.

One honest limitation, recorded rather than hidden: the middleware cannot
commit its record inside the handler's transaction, because the repository
ports beneath it are per-operation. If a write commits and its record does
not, a later retry re-executes. That is today's behaviour, so the mechanism
is never a regression — but it is why this is a `warn!` in the logs and a
paragraph here.

Standalone `dpp-vault` and standalone `dpp-integrator` do not mount it: the
former spawns no purge task and the latter has no database, and a key store
that only grows, or that forgets on restart, is worse than none.

- **A continuity snapshot now says how long it stands, under its own signature.**
Each snapshot written to object storage carries `asOf`, `validUntil` and a
`snapshotJwsSignature` over the whole document — the passport's public view,
Expand Down
8 changes: 8 additions & 0 deletions CLAUDE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -314,6 +314,14 @@ Background cleanup task runs every 6 hours, deleting completed/failed jobs older
> in the handler, not by the middleware**, so they are the column most likely to
> go stale — verify against the handler's first lines before relying on a row.

**Which routes accept an `Idempotency-Key` is deliberately not a column here.**
That set is decided in one place —
`dpp_common::idempotency::policy`'s `KEYED` table — and asserted against the
live routers by `every_keyed_route_is_a_route_the_node_serves` in
`crates/dpp-node/tests/openapi_contract.rs`. A second copy in this table would
be a second thing to drift, and a keyed route missing from it would read as an
unprotected one. Read the table in the code.

### MVP Node (port 8001)

| Method | Path | Auth | Handler |
Expand Down
33 changes: 33 additions & 0 deletions api/components/parameters/IdempotencyKey.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
name: Idempotency-Key
in: header
required: false
description: |
An opaque, client-minted string that makes this write safe to retry.

Send the same key with the same body and the first outcome is returned
instead of a second resource being created. The replayed response carries
`Idempotency-Replayed: true`, so a client can tell a replay from a first
execution without comparing anything.

Only the routes that document this parameter accept it. Sending it to any
other route is a `400` rather than being ignored — a route that is idempotent
by shape records nothing, and silently accepting the header would suggest a
protection that is not there.

**A retry must resend byte-identical bytes.** The key is bound to a SHA-256
of the raw request body, not of a canonicalised form, so re-serialising with
different member order or whitespace counts as a different request and is
refused with `422`. Use a fresh key whenever the body changes.

Keys are scoped to the authenticated caller and to this route, so the same
key may safely be reused across different operations, and one caller can
never observe another's. A key is honoured for **24 hours**; after that the
same key starts a new request.

While a first attempt is still running, a duplicate receives `409` with
`Retry-After`.
schema:
type: string
minLength: 1
maxLength: 255
example: 6f2a1c40-2f6f-4b8a-9a3e-1f4c9b2d7e51
26 changes: 26 additions & 0 deletions api/components/responses/IdempotentRequestInFlight.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
description: |
An earlier attempt carrying this `Idempotency-Key` is still running.

Returned instead of letting a duplicate execute alongside it. Retry shortly
with the same key and the same body; `Retry-After` carries the suggested
delay.

A claim left behind by a process that died mid-request is reclaimed
automatically after 60 seconds, so this can never wedge a key permanently.
headers:
Retry-After:
description: Seconds to wait before retrying.
schema:
type: integer
minimum: 1
content:
application/problem+json:
schema:
$ref: ../schemas/errors/Problem.yaml
example:
type: https://problems.odal-node.io/idempotent-request-in-flight
title: Idempotent Request In Flight
status: 409
detail: >-
An earlier attempt with this `Idempotency-Key` is still running. Retry
shortly; do not change the body.
104 changes: 100 additions & 4 deletions api/openapi.bundled.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,11 @@
"BasicAuth": []
}
],
"parameters": [
{
"$ref": "#/components/parameters/IdempotencyKey"
}
],
"requestBody": {
"required": true,
"content": {
Expand Down Expand Up @@ -178,6 +183,9 @@
"403": {
"$ref": "#/components/responses/Forbidden"
},
"409": {
"$ref": "#/components/responses/IdempotentRequestInFlight"
},
"422": {
"$ref": "#/components/responses/ValidationError"
}
Expand Down Expand Up @@ -1269,6 +1277,9 @@
"schema": {
"$ref": "#/components/schemas/DppId"
}
},
{
"$ref": "#/components/parameters/IdempotencyKey"
}
],
"responses": {
Expand Down Expand Up @@ -1798,7 +1809,7 @@
"post": {
"operationId": "addFacility",
"summary": "Add a facility",
"description": "Add a facility. The identifier is validated by scheme — a `gln` must\npass the GS1 mod-10 check digit. Requires an admin-scoped key.\n",
"description": "Add a facility. The identifier is validated by scheme — a `gln` must\npass the GS1 mod-10 check digit. Requires an admin-scoped key.\n\nFacilities are retired, never deleted, so a duplicate created by a retried\nrequest cannot be cleaned up afterwards. Send an `Idempotency-Key`.\n",
"tags": [
"Facilities"
],
Expand All @@ -1810,6 +1821,11 @@
"BasicAuth": []
}
],
"parameters": [
{
"$ref": "#/components/parameters/IdempotencyKey"
}
],
"requestBody": {
"required": true,
"content": {
Expand Down Expand Up @@ -1837,6 +1853,9 @@
"403": {
"$ref": "#/components/responses/Forbidden"
},
"409": {
"$ref": "#/components/responses/IdempotentRequestInFlight"
},
"422": {
"$ref": "#/components/responses/ValidationError"
}
Expand Down Expand Up @@ -2032,7 +2051,7 @@
"post": {
"operationId": "addOperatorIdentifier",
"summary": "Add an operator identifier",
"description": "Add an economic-operator identifier. Validated by scheme — LEI uses\nISO 7064 MOD 97-10; DUNS is 9 digits; EORI/VAT require a country prefix.\nRequires an admin-scoped key.\n",
"description": "Add an economic-operator identifier. Validated by scheme — LEI uses\nISO 7064 MOD 97-10; DUNS is 9 digits; EORI/VAT require a country prefix.\nRequires an admin-scoped key.\n\nOperator identifiers are retired, never deleted, so a duplicate created by\na retried request cannot be cleaned up afterwards. Send an\n`Idempotency-Key`.\n",
"tags": [
"Operator Identifiers"
],
Expand All @@ -2044,6 +2063,11 @@
"BasicAuth": []
}
],
"parameters": [
{
"$ref": "#/components/parameters/IdempotencyKey"
}
],
"requestBody": {
"required": true,
"content": {
Expand Down Expand Up @@ -2071,6 +2095,9 @@
"403": {
"$ref": "#/components/responses/Forbidden"
},
"409": {
"$ref": "#/components/responses/IdempotentRequestInFlight"
},
"422": {
"$ref": "#/components/responses/ValidationError"
}
Expand Down Expand Up @@ -2266,7 +2293,7 @@
"post": {
"operationId": "createApiKey",
"summary": "Create a new API key",
"description": "Generate a new API key. The response includes the full plain-text\n`secret` — this is the only time it will be shown. Store it securely.\n",
"description": "Generate a new API key. The response includes the full plain-text\n`secret` — this is the only time it will be shown. Store it securely.\n\n**A replayed request does not return the secret.** This route accepts an\n`Idempotency-Key`, but the secret is never stored, so a retry that finds a\ncompleted key answers with the key record and\n`\"secretAlreadyDelivered\": true` in place of `secret`. That is the honest\nanswer: the credential exists and was handed to the first attempt. If that\nresponse was lost, revoke the key and create another.\n",
"tags": [
"API Keys"
],
Expand All @@ -2278,6 +2305,11 @@
"BasicAuth": []
}
],
"parameters": [
{
"$ref": "#/components/parameters/IdempotencyKey"
}
],
"requestBody": {
"required": true,
"content": {
Expand Down Expand Up @@ -2305,6 +2337,9 @@
"403": {
"$ref": "#/components/responses/Forbidden"
},
"409": {
"$ref": "#/components/responses/IdempotentRequestInFlight"
},
"422": {
"$ref": "#/components/responses/ValidationError"
}
Expand Down Expand Up @@ -2405,7 +2440,7 @@
"post": {
"operationId": "createWebhook",
"summary": "Create a webhook subscription",
"description": "Register a receiver URL (must be https and resolve to a public host,\nunless the node sets WEBHOOK_ALLOW_PRIVATE_TARGETS). The response includes\nthe `secret` — shown ONCE — used to verify the `X-Odal-Signature` header\non every delivery. Admin-scoped.\n",
"description": "Register a receiver URL (must be https and resolve to a public host,\nunless the node sets WEBHOOK_ALLOW_PRIVATE_TARGETS). The response includes\nthe `secret` — shown ONCE — used to verify the `X-Odal-Signature` header\non every delivery. Admin-scoped.\n\n**A replayed request does not return the secret.** This route accepts an\n`Idempotency-Key`, but the signing secret is never stored, so a retry that\nfinds a completed key answers with the subscription and\n`\"secretAlreadyDelivered\": true` in place of `secret`. If the first\nresponse was lost, delete the subscription and create another.\n",
"tags": [
"Webhooks"
],
Expand All @@ -2417,6 +2452,11 @@
"BasicAuth": []
}
],
"parameters": [
{
"$ref": "#/components/parameters/IdempotencyKey"
}
],
"requestBody": {
"required": true,
"content": {
Expand Down Expand Up @@ -2444,6 +2484,9 @@
"403": {
"$ref": "#/components/responses/Forbidden"
},
"409": {
"$ref": "#/components/responses/IdempotentRequestInFlight"
},
"422": {
"$ref": "#/components/responses/ValidationError"
}
Expand Down Expand Up @@ -2565,6 +2608,11 @@
"BasicAuth": []
}
],
"parameters": [
{
"$ref": "#/components/parameters/IdempotencyKey"
}
],
"requestBody": {
"required": true,
"content": {
Expand Down Expand Up @@ -2616,6 +2664,9 @@
"403": {
"description": "A non-admin credential attempted to install a plugin."
},
"409": {
"$ref": "#/components/responses/IdempotentRequestInFlight"
},
"422": {
"description": "The artifact was rejected — bad signature, incompatible ABI, or a non-instantiable/incompatible module."
},
Expand Down Expand Up @@ -3280,6 +3331,9 @@
"type": "string",
"example": "battery"
}
},
{
"$ref": "#/components/parameters/IdempotencyKey"
}
],
"requestBody": {
Expand Down Expand Up @@ -3336,6 +3390,9 @@
"404": {
"description": "Unknown product group."
},
"409": {
"$ref": "#/components/responses/IdempotentRequestInFlight"
},
"422": {
"$ref": "#/components/responses/ValidationError"
}
Expand Down Expand Up @@ -7502,6 +7559,20 @@
}
}
},
"parameters": {
"IdempotencyKey": {
"name": "Idempotency-Key",
"in": "header",
"required": false,
"description": "An opaque, client-minted string that makes this write safe to retry.\n\nSend the same key with the same body and the first outcome is returned\ninstead of a second resource being created. The replayed response carries\n`Idempotency-Replayed: true`, so a client can tell a replay from a first\nexecution without comparing anything.\n\nOnly the routes that document this parameter accept it. Sending it to any\nother route is a `400` rather than being ignored — a route that is idempotent\nby shape records nothing, and silently accepting the header would suggest a\nprotection that is not there.\n\n**A retry must resend byte-identical bytes.** The key is bound to a SHA-256\nof the raw request body, not of a canonicalised form, so re-serialising with\ndifferent member order or whitespace counts as a different request and is\nrefused with `422`. Use a fresh key whenever the body changes.\n\nKeys are scoped to the authenticated caller and to this route, so the same\nkey may safely be reused across different operations, and one caller can\nnever observe another's. A key is honoured for **24 hours**; after that the\nsame key starts a new request.\n\nWhile a first attempt is still running, a duplicate receives `409` with\n`Retry-After`.\n",
"schema": {
"type": "string",
"minLength": 1,
"maxLength": 255
},
"example": "6f2a1c40-2f6f-4b8a-9a3e-1f4c9b2d7e51"
}
},
"responses": {
"Unauthorized": {
"description": "Missing or invalid authentication credentials.",
Expand Down Expand Up @@ -7535,6 +7606,31 @@
}
}
},
"IdempotentRequestInFlight": {
"description": "An earlier attempt carrying this `Idempotency-Key` is still running.\n\nReturned instead of letting a duplicate execute alongside it. Retry shortly\nwith the same key and the same body; `Retry-After` carries the suggested\ndelay.\n\nA claim left behind by a process that died mid-request is reclaimed\nautomatically after 60 seconds, so this can never wedge a key permanently.\n",
"headers": {
"Retry-After": {
"description": "Seconds to wait before retrying.",
"schema": {
"type": "integer",
"minimum": 1
}
}
},
"content": {
"application/problem+json": {
"schema": {
"$ref": "#/components/schemas/Problem"
},
"example": {
"type": "https://problems.odal-node.io/idempotent-request-in-flight",
"title": "Idempotent Request In Flight",
"status": 409,
"detail": "An earlier attempt with this `Idempotency-Key` is still running. Retry shortly; do not change the body."
}
}
}
},
"ValidationError": {
"description": "One or more fields failed validation.",
"content": {
Expand Down
Loading
Loading