Skip to content

feat: add server-side fallbacks - #32

Merged
sophie-ant merged 1 commit into
mainfrom
sdownward/server-side-fallbacks
Sep 9, 2026
Merged

sophie-ant merged 1 commit into
mainfrom
sdownward/server-side-fallbacks

Conversation

@sophie-ant

@sophie-ant sophie-ant commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds server-side fallbacks. Some models decline requests in certain policy areas, such as cybersecurity or biology. With fallbacks: on ClaudeLanguageModel, the API retries a declined request on substitute models within the same request, and the response continues where the declining model stopped:

ClaudeLanguageModel(name: .opus5, auth: auth, fallbacks: [.opus4_8])

This follows the server-side fallbacks support in the TypeScript and Java SDKs, and it uses the same stream handling as their message accumulators.

  • Request: fallbacks takes up to three models, or .serverDefault for the model's default configuration.
  • Betas: a list of fallbacks sends server-side-fallback-2026-06-01. .serverDefault sends server-side-fallback-2026-07-01, which it needs. Each form sends the narrowest value that admits it. The beta is added after any anthropic-beta values that a .proxied relay sends.
  • Per-fallback fields: each fallback gets the thinking and effort it accepts. An entry's field replaces the request's whole field, so an entry sends complete values: a fallback that doesn't think gets thinking turned off, and an entry's output_config keeps the request's schema. With fixedEffort:, each fallback gets the closest level it accepts. A fallback entry can't change sampling, a schema, or images, so those need every model in the chain.
  • Replay: fallback blocks are recorded and replayed in place, because the API rejects the thinking before a handover if the block is missing. A request that replays one also sends the beta, even without fallbacks, so a transcript from a fallback session stays usable.
  • Accessors: claudeHandovers, claudeHandover(for:), and claudeModelID. A handover is a ClaudeModelHandover. After a handover, the API routes the conversation straight to the fallback model for a while, and those responses carry no handover block. claudeModelID still reports the model that served them.

Out of scope: client-side fallbacks, like the TypeScript SDK's refusal middleware, for platforms without server-side support. This package talks to the Anthropic API directly.

Test plan

Live requests in the shapes this sends, on claude-opus-5 with claude-opus-4-8 as the fallback, under server-side-fallback-2026-06-01:

  • A list of fallbacks returns 200. Without a fallbacks beta, it returns 400.
  • Entries with complete overrides return 200: thinking: {"type": "disabled"}, output_config: {}, and an output_config that carries the schema, with or without an effort.
  • A replayed fallback block returns 200 with the beta, and 400 without it.
  • A merged anthropic-beta header returns 200.
  • "default" returns 400 under this beta, and 200 under server-side-fallback-2026-07-01.

I couldn't trigger a real handover with a benign prompt. The handover path is covered by unit tests, using a stream fixture in the shape the other SDKs handle.

The full test suite at 7ceb16a, via xcodebuild on macOS 26.6 with Xcode 27.0 beta 6 (27A5252f) and the iOS 27.0 simulator: ** TEST SUCCEEDED **, 240 tests, 237 passed, 0 failed, 3 skipped. The 3 skips are the keychain-backed AppAttestStoreTests, which skip by design under this runner.

Some models decline requests in certain policy areas. With `fallbacks:`
on `ClaudeLanguageModel`, the API retries a declined request on
substitute models within the same request, and the response continues
where the declining model stopped:

    ClaudeLanguageModel(name: .opus5, auth: auth, fallbacks: [.opus4_8])

- `fallbacks` takes up to three models, or `.serverDefault` for the
  model's default configuration. A list sends the
  `server-side-fallback-2026-06-01` beta, and `.serverDefault` sends
  `server-side-fallback-2026-07-01`, which it needs. The beta is added
  after any `anthropic-beta` values a `.proxied` relay already sends.
- Each fallback gets the thinking and effort it accepts. An entry's
  field replaces the request's whole field, so an entry sends complete
  values: a fallback that doesn't think gets thinking turned off, and
  its `output_config` keeps the request's schema. With `fixedEffort:`,
  each fallback gets the closest level it accepts. Sampling, a schema,
  and images can't differ per entry, so those need every model in the
  chain.
- `fallback` blocks are recorded and replayed in place, because the
  API rejects the thinking before a handover if the block is missing. A
  request that replays one sends the beta even without `fallbacks`, so
  a transcript from a fallback session stays usable.
- `ClaudeModelHandover`, `claudeHandovers`, `claudeHandover(for:)`, and
  `claudeModelID` report handovers and the model that served each
  response. After a handover the API routes the conversation straight
  to the fallback model for a while; those responses carry no block,
  and `claudeModelID` still names the model.
@sophie-ant
sophie-ant merged commit 9eae924 into main Sep 9, 2026
2 checks passed
@sophie-ant
sophie-ant deleted the sdownward/server-side-fallbacks branch September 9, 2026 17:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants