Skip to content

feat(agent-scan): support custom transforms for SSE responses - #600

Open
ichaobuster wants to merge 1 commit into
Tencent:mainfrom
ichaobuster:codex/sse-transform-response
Open

feat(agent-scan): support custom transforms for SSE responses#600
ichaobuster wants to merge 1 commit into
Tencent:mainfrom
ichaobuster:codex/sse-transform-response

Conversation

@ichaobuster

@ichaobuster ichaobuster commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Problem

Custom HTTP agent providers can configure transform_response to extract output from non-standard response payloads. For SSE responses, however, each event was normalized before _extract_output() applied the transform. A transform written for the original event shape (for example, stream.text) therefore could not extract the streaming content.

Changes

  • pass transform_response into the SSE parser and apply it to each JSON data: event
  • accumulate transformed content separately from the existing fallback parsers
  • use transformed content exclusively when at least one event matches, avoiding mixed output from unrelated event shapes
  • preserve the existing OpenAI, Anthropic, Coze, Dify, and plain-text parsing when no event matches the transform
  • normalize transformed streams through content with raw_sse: false
  • preserve token-usage extraction

Behavior

  • Transform matches: concatenate only transformed event content.
  • Transform does not match any event: fall back to the existing SSE parsing behavior.
  • Mixed event shapes: ignore fallback content once the configured transform succeeds, preventing status or metadata events from contaminating the response.
  • Custom events containing type: keep the generic transformed response shape instead of treating them as Anthropic events.

Scope

This change only affects SSE handling for HTTP providers with an optional response transform. Non-SSE responses and existing provider-specific parsers remain unchanged.

Testing

  • python -m pytest -vv pytests/test_agent_adapter.py — 4 passed
  • Python syntax compilation — passed
  • git diff --check — passed

The unit tests cover HTTP integration, transform aggregation, complete fallback, mixed event shapes, token usage, and custom events containing a top-level type field.

@ichaobuster ichaobuster changed the title Support custom transforms for SSE streaming responses feat(agent-scan): Support custom transforms for SSE streaming responses Aug 25, 2026
@ichaobuster ichaobuster changed the title feat(agent-scan): Support custom transforms for SSE streaming responses feat(agent-scan): support custom transforms for SSE responses Aug 25, 2026
@boy-hack

Copy link
Copy Markdown
Collaborator

Thanks @ichaobuster. Clean fix for a real gap: SSE events were normalized before transform_response ran, so a transform written for the raw event shape couldn't extract streaming content. Applying the transform per-event and switching the whole output to the transformed path once at least one event matches is the right approach.

Verification I did against the branch:

  • token_usage is still captured via the bottom if "usage" in data and data["usage"] regardless of the transform branch → token extraction is preserved. Good.
  • The elif "choices" / elif "type" chain correctly excludes fallback parsers once a transform matched (transform_applied), preventing status/metadata events from contaminating output — matches the "mixed event shapes" requirement.
  • 4 new unit tests cover match / fallback / mixed / custom-type. Nice coverage.

Two minor, non-blocking points:

  1. raw_sse: False when transform matches (vs True in the fallback path): presenting a clean content shape makes sense, but double-check no downstream consumer keys behavior on raw_sse in a way that breaks the transform case. Tests pass, so likely fine — just flagging.
  2. Empty-string transform edge: if the transform returns "" for every event (not None), transform_applied stays True, full_content becomes "", and you lose the fallback data. Acceptable given "transform wins", but consider if transformed (truthy) vs is not None if empty-string events should fall back.

LGTM overall — the test coverage is the part I'd most want to keep.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants