fix(kms): ship CA certificates in the KMS image

[kvinwang]

Problem

dstack-kms does not start in its own published image. Running dstacktee/dstack-kms (here cr.kvin.wang/dstack/dstack-kms:0.6.0-rc0, built from cb4c3736c5) with a stock config dies during onboarding, before it serves anything:

INFO dstack_kms: Onboarding
Error: failed to load attestation verifier

Caused by:
    0: builder error
    1: unexpected error: No CA certificates were loaded from the system

The failure is not about attestation trust anchors — those are already compiled in and behave correctly. AttestationVerifier::load falls back to the built-in Intel root and AMD ARKs when no override is configured (dstack/dstack-attest/src/attestation.rs:96-99, :110). What is missing is the web PKI root store used for the TLS transport to the collateral endpoints.

The chain:

1. AttestationVerifier::load eagerly constructs both collateral clients — CollateralClient::with_default_http(pccs) and AmdKdsClient::with_base_url(amd_kds) (dstack/dstack-attest/src/attestation.rs:147,152).
2. AmdKdsClient::with_base_url builds a reqwest::Client (dstack/sev-snp-qvl/src/lib.rs:317-321).
3. The workspace pins reqwest with default-features = false and the rustls feature (dstack/Cargo.toml:214-220), which pulls in rustls-platform-verifier. Cargo.lock has rustls-native-certs and no webpki-roots, so the only root source is the system trust store.
4. rustls-platform-verifier returns Err when that store comes up empty (rustls-platform-verifier-0.7.0/src/verification/others.rs:104-108).
5. The KMS runtime stage was bare debian:bookworm plus the binary, and the Debian base image ships no ca-certificates.

Two details make this worse than it looks. reqwest fails when the client is built, not when a request is made, so the process dies even though nothing has tried to reach the network yet. And the client that kills it is the AMD KDS one — an Intel TDX deployment that will never talk to AMD KDS still cannot boot.

The sibling image already gets this right: dstack-gateway installs ca-certificates in its runtime stage (dstack/gateway/dstack-app/builder/Dockerfile:40) and verifies a runtime package list in CI. The KMS had neither, and git log -S ca-certificates -- dstack/kms/dstack-app/builder/Dockerfile is empty, so it has been this way since the image was introduced rather than being a recent regression.

Fix

Install ca-certificates in the KMS runtime stage, following the gateway's pattern exactly rather than inventing a second one:

• Dockerfile — run pin-packages.sh against a runtime pinned-packages.txt and install ca-certificates from the same frozen Debian snapshot the gateway uses. WORKDIR is restored to / afterwards so the image's working directory is unchanged.
• build-image.sh — extract the runtime package list too, mirroring gateway/dstack-app/builder/build-image.sh:31,33. Without this the new list is never regenerated and check_clean_tree cannot notice drift.
• docker-build-check.yml — verify the runtime list for the KMS image. CI previously checked only the KMS builder list while checking both for the gateway; that gap is part of why this went unnoticed.
• shared/pinned-packages.txt — generated by build-image.sh from the built image, not hand-written. It resolves ca-certificates=20230311+deb12u1, the same version the gateway already pins, so both images stay on one snapshot.

This keeps the image reproducible: the package set is pinned to exact versions from the frozen snapshot, and CI now fails if it drifts.

Fixing it in the image rather than the code is the minimal change that makes the published artifact work and brings the two images into line. Bundling webpki-roots into the binary, or constructing the collateral clients lazily so an Intel-only deployment never builds the AMD one, are both defensible and orthogonal — worth considering separately, but neither is needed to make the image boot.

Verification

A/B on one host, same config, neither container given a host CA mount — the only variable is the image:

Image	Result
this PR (dstack-kms:ca-fix)	Up, rocket::rkt: endpoint=https://0.0.0.0:8000 (TCP)
released 0.6.0-rc0 (control)	Exited (1), No CA certificates were loaded from the system

The control run is what pins the cause to this change rather than to some other difference in the environment.

Beyond starting, the fixed image completes onboarding and serves RPC:

$ curl -kfsS https://127.0.0.1:9401/prpc/GetMeta?json
  ca_cert 存在: True
  k256_pubkey: 0252113ca2ea14c62c221d09...

and writes the expected key material — root-ca.crt, root-ca.key, rpc.crt, rpc.key, rpc-domain.

Image contents, before and after:

dstack-kms:ca-fix              /etc/ssl/certs/ca-certificates.crt   216591 bytes
cr.kvin.wang/.../0.6.0-rc0     /etc/ssl/certs/ca-certificates.crt   MISSING

216591 bytes is byte-identical in size to the bundle the gateway image already ships, as expected from the shared snapshot and pin.

Both build-image.sh runs completed with check_clean_tree clean on the second pass, confirming the generated pinned-packages.txt is stable.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.