fix: drain SSE buffers and key OpenAI tool-call index

[undivisible]

Summary

• Portkey and Cloudflare SSE parsers re-scanned the entire accumulated buffer on every chunk, re-emitting earlier TextDeltas (and [DONE]) with quadratic cost. They now drain completed newline-terminated frames the same way xAI already does.
• OpenAI-compatible streaming keyed pending tool calls by choice index, so parallel delta.tool_calls[].index values collided. Parser now uses the tool-call index, with serde defaults for omitted delta fields (id/role/function.name).
• CacheMiddleware includes max_steps in the cache key so agent-loop depth cannot return a cached one-shot result.

Tests

• split_sse_chunks_emit_each_delta_once on Portkey and Cloudflare
• parallel_tool_calls_use_tool_index_not_choice_index
• different_max_steps_is_a_cache_miss
• cargo test --workspace --all-features

Notes

Backward-compatible. Does not touch OAuth/auth. Safe to auto-merge after CI is green.

Made with Amp

---

Note

Medium Risk
Streaming and parallel tool-call parsing changes affect live provider behavior; cache key change only affects correctness of cached agent-loop results, not security.

Overview
Fixes SSE streaming for Cloudflare and Portkey: instead of re-parsing the full accumulated buffer on every chunk (which duplicated TextDelta events and scaled poorly), each chunk now drains only newline-complete frames via parse_stream_chunk, matching the incremental pattern used elsewhere.

For OpenAI-compatible streaming, pending tool calls are keyed by delta.tool_calls[].index (with choice index as fallback), so parallel tool streams no longer collide. API types gain serde(default) (and optional index) so partial tool-call deltas deserialize cleanly; outbound tool calls set index: None.

CacheMiddleware now hashes max_steps into the cache key so different agent loop depths cannot share a cached response.

Adds targeted tests for split SSE chunks, parallel tool-call indexing, and max_steps cache misses.

^{Reviewed by Cursor Bugbot for commit ba5661c. Configure here.}

[cursor]

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: serverGenReqId_467e4a13-5409-4727-8fc6-566c7e78dbba)

[mergify]

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

• Queue this pull request