Skip to content

test(providers): share the OpenAI-compatible contract across thin provider tests - #983

Open
SantiagoDePolonia wants to merge 2 commits into
test/shared-test-helpersfrom
test/testify-providers-b
Open

test(providers): share the OpenAI-compatible contract across thin provider tests#983
SantiagoDePolonia wants to merge 2 commits into
test/shared-test-helpersfrom
test/testify-providers-b

Conversation

@SantiagoDePolonia

Copy link
Copy Markdown
Contributor

Rewrites the tests of the thin provider adapters (every internal/providers/<name> package except anthropic, gemini, and openai) on top of providertest and testify.

  • Fifteen providers built on the OpenAI-compatible adapter (hetzner, kilo, kimicode, meta, fireworks, deepseek, zai, xiaomi, minimax, opencodego, chutes, groq, bailian, and others) now run one TestChatCompatibleContract via providertest.AssertChatCompatible, which covers registration metadata, constructor safety, chat, streaming, model listing, Responses translation, and embeddings. The per-package copies of those tests are removed; provider-specific tests (headers, reasoning mapping, model ID rewriting, audio, images, realtime, error mapping) stay.
  • Providers whose Responses path posts natively to /responses (oracle, sglang, vllm, llamacpp, llmd, openrouter, xai) and the azure and bedrockmantle adapters keep their own tests and are noted in the report.
  • Constructor non-nil, does-not-panic, and compile-time interface tests removed; interface checks become package-level var _ declarations. Duplicate cases merged into tables (ollama, googlecommon, chatgpt, vllm, deepseek).

Test-only change, about 5,600 net lines removed. Stacked on #976 (helper packages).

@coderabbitai

coderabbitai Bot commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

Next included review available in 11 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 914407b0-be90-4d96-a852-e8407be77d5d

📥 Commits

Reviewing files that changed from the base of the PR and between 7192b3a and 14df4f5.

📒 Files selected for processing (54)
  • internal/providers/azure/azure_test.go
  • internal/providers/azure/realtime_test.go
  • internal/providers/bailian/bailian_test.go
  • internal/providers/bailian/realtime_test.go
  • internal/providers/bedrock/bedrock_test.go
  • internal/providers/bedrockmantle/bedrock_mantle_test.go
  • internal/providers/bedrockmantle/config_test.go
  • internal/providers/chatgpt/chatgpt_test.go
  • internal/providers/chutes/chutes_test.go
  • internal/providers/chutes/models_test.go
  • internal/providers/cohere/audio_test.go
  • internal/providers/cohere/cohere_test.go
  • internal/providers/cohere/passthrough_semantics_test.go
  • internal/providers/deepseek/deepseek_test.go
  • internal/providers/deepseek/passthrough_semantics_test.go
  • internal/providers/elevenlabs/audio_test.go
  • internal/providers/elevenlabs/elevenlabs_test.go
  • internal/providers/fireworks/fireworks_test.go
  • internal/providers/fireworks/reasoning_test.go
  • internal/providers/googlecommon/auth_test.go
  • internal/providers/groq/audio_test.go
  • internal/providers/groq/groq_test.go
  • internal/providers/groq/reasoning_response_test.go
  • internal/providers/groq/reasoning_test.go
  • internal/providers/health/tracker_test.go
  • internal/providers/hetzner/hetzner_test.go
  • internal/providers/kilo/kilo_test.go
  • internal/providers/kilo/passthrough_semantics_test.go
  • internal/providers/kimicode/kimicode_test.go
  • internal/providers/llamacpp/llamacpp_test.go
  • internal/providers/llamacpp/models_test.go
  • internal/providers/llmd/llmd_test.go
  • internal/providers/meta/meta_test.go
  • internal/providers/minimax/audio_test.go
  • internal/providers/minimax/minimax_test.go
  • internal/providers/ollama/ollama_test.go
  • internal/providers/opencodego/opencodego_test.go
  • internal/providers/opencodego/reasoning_test.go
  • internal/providers/opencodego/session_test.go
  • internal/providers/openrouter/openrouter_test.go
  • internal/providers/openrouter/passthrough_semantics_test.go
  • internal/providers/oracle/oracle_test.go
  • internal/providers/sglang/sglang_test.go
  • internal/providers/vertex/vertex_test.go
  • internal/providers/vllm/reasoning_test.go
  • internal/providers/vllm/vllm_test.go
  • internal/providers/xai/images_test.go
  • internal/providers/xai/realtime_test.go
  • internal/providers/xai/xai_test.go
  • internal/providers/xiaomi/audio_test.go
  • internal/providers/xiaomi/xiaomi_test.go
  • internal/providers/zai/passthrough_semantics_test.go
  • internal/providers/zai/realtime_test.go
  • internal/providers/zai/zai_test.go

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@greptile-apps

greptile-apps Bot commented Sep 12, 2026

Copy link
Copy Markdown

RetriggerConfidence Score: 4/5

Safe to merge with a non-blocking test-coverage concern in the vLLM reasoning adapter test.

Reviews (1) · Last reviewed commit: "test(providers): share the OpenAI-compat..."

t.Fatal("adaptChatRequest() copied a request it didn't need to change")
}
require.NoError(t, err)
assert.Equal(t, req, adapted)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Preserve Request Identity

This no-op test uses structural equality, so it also passes when the adapter allocates and returns an identical copy of the request. The implementation returns the original request when no legacy reasoning content needs adaptation; use require.Same(t, req, adapted) (or assert.Same) so an unnecessary-copy regression is caught. This is a non-blocking test-coverage concern, but retaining the pointer assertion avoids silently changing the no-op contract.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Artifacts

Evidence from the check

  • The exact temporary in-package Go test was saved before removal; it checks the real no-op pointer plus the deep-equality-versus-identity distinction, proving the required assertion semantics.

Command output from the check

  • The original focused vLLM test command completed successfully with exit code 0, showing the current structural assertion passes but not establishing pointer identity.

Command output from the check

  • The authored focused Go test completed successfully with exit code 0, proving the current no-op implementation returns the original request pointer and that the stronger assertion is executable.

Command output from the check

  • Captured numbered source evidence shows the deep-equality assertion, the implementation’s conditional copy and `return req`, and that DeepSeek and MiniMax tests are included in this commit’s changed diff, confirming the candidate.

View artifacts

T-Rex Ran code and verified through T-Rex

@SantiagoDePolonia

Copy link
Copy Markdown
Contributor Author

The vllm no-op adaptation checks now use assert.Same (14df4f5), along with the other sites in this PR where the original compared request pointers (bailian, deepseek, minimax).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant