ChatOpenAI (Responses API) mutates model_kwargs["text"] in place: one JSON mode call permanently forces JSON mode on later plain calls

[Harshxth]

Submission checklist

• This is a bug, not a usage question.
• I added a clear and descriptive title that summarizes this issue.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).
• This is not related to the langchain-community package.
• I posted a self-contained, minimal, reproducible example. A maintainer can copy it and run it AS IS.

Package (Required)

• langchain
• langchain-openai
• langchain-anthropic
• langchain-classic
• langchain-core
• langchain-model-profiles
• langchain-tests
• langchain-text-splitters
• langchain-chroma
• langchain-deepseek
• langchain-exa
• langchain-fireworks
• langchain-groq
• langchain-huggingface
• langchain-mistralai
• langchain-nomic
• langchain-ollama
• langchain-openrouter
• langchain-perplexity
• langchain-qdrant
• langchain-xai
• Other / not sure / general

Related Issues / PRs

• ChatPerplexity Responses route mutates the caller's extra_body dict in place — per-call params leak into the instance and all later requests #38840 reports the same bug class (in-place mutation of a caller dict on the Responses route) in langchain-perplexity.
• ChatOpenAI (Responses API) mutates model_kwargs["text"] in place: one JSON mode call permanently forces JSON mode on later plain calls #38868 was this same report; it was auto-closed because the package checkbox failed to register on submit, so refiling with the package selected.

Reproduction Steps / Example Code (Python)

from langchain_openai import ChatOpenAI

# Caller-owned options dict, passed by reference via model_kwargs.
my_text_opts = {"verbosity": "low"}

llm = ChatOpenAI(
    model="gpt-4.1",
    api_key="test",
    use_responses_api=True,
    model_kwargs={"text": my_text_opts},
)

print("caller dict BEFORE any call:        ", my_text_opts)

# Call 1: a JSON mode / structured output request (what with_structured_output
# with method="json_mode" binds: response_format={"type": "json_object"}).
# Offline: only builds the request payload, never touches the network.
payload1 = llm._get_request_payload(
    "give me json", response_format={"type": "json_object"}
)
print("payload1['text'] (expected format): ", payload1["text"])

print("caller dict AFTER structured call:  ", my_text_opts)
print("llm.model_kwargs AFTER:             ", llm.model_kwargs)

# Call 2: a PLAIN call, no response_format anywhere.
payload2 = llm._get_request_payload("just chat normally")
print("payload2['text'] (plain call!):     ", payload2["text"])

assert "format" not in my_text_opts, "caller's dict was mutated in place"

Error Message and Stack Trace (if applicable)

Observed output (langchain-openai 1.3.5 and current master):

caller dict BEFORE any call:         {'verbosity': 'low'}
payload1['text'] (expected format):  {'verbosity': 'low', 'format': {'type': 'json_object'}}
caller dict AFTER structured call:   {'verbosity': 'low', 'format': {'type': 'json_object'}}
llm.model_kwargs AFTER:              {'text': {'verbosity': 'low', 'format': {'type': 'json_object'}}}
payload2['text'] (plain call!):      {'verbosity': 'low', 'format': {'type': 'json_object'}}
AssertionError: caller's dict was mutated in place

Description

With use_responses_api=True, _construct_responses_api_payload writes the response format and verbosity straight into payload["text"]:

• payload["text"]["format"] = {"type": "json_object"} (JSON mode)
• payload["text"]["format"] = format_value (json_schema)
• payload["text"]["verbosity"] = verbosity

(libs/partners/openai/langchain_openai/chat_models/base.py, around lines 4356-4383 on current master.)

The problem is that payload is built by shallow-merging self.model_kwargs (and per-call kwargs), so payload["text"] is the very same dict object the caller passed as model_kwargs={"text": {...}}. Those writes therefore land in the instance's model_kwargs and in the caller's own dict.

What I expect: request payload construction should not modify the model instance or the caller's inputs. Each call's text.format should apply to that call only.

What happens instead: after one JSON mode or structured output call, model_kwargs["text"] permanently contains format: {"type": "json_object"}, so every later plain call on the same instance is silently forced into JSON mode (see payload2 in the repro, which never asked for any response format). The caller's dict is also corrupted, so sharing one options dict across two ChatOpenAI instances spreads the pollution.

The fix looks small: copy the text dict once during payload construction before writing into it, mirroring what was done for the same bug class in langchain-perplexity (#38840). I have the fix and a regression test ready. I'd like to work on this, could you assign it to me?

System Info

System Information

OS: Windows
OS Version: 10.0.26200
Python Version: 3.11.0 (main, Oct 24 2022, 18:26:48) [MSC v.1933 64 bit (AMD64)]

Package Information

langchain_core: 1.4.9
langchain: 1.3.13
langsmith: 0.10.5
langchain_openai: 1.3.5
langgraph_sdk: 0.4.2

Also reproduced against current master (commit 7bf8fe2), where the same in-place writes are present in _construct_responses_api_payload.

[gitbalaji]

Hi, I'd like to work on this. Could a maintainer please assign it to me?

[zerafachris]

I'd like to work on this fix. Can I be assigned?

[zerafachris]

I have a fix ready (copy the text dict before mutation in _construct_responses_api_payload). Could a maintainer please assign me so I can submit the PR? The fix is 4 lines — same pattern already applied in langchain-perplexity (#38840).

[zeweihan]

Great repro, and the "one JSON call permanently forces JSON mode on every later plain call" framing is the exact part worth flagging, because the failure shape is cross-call contamination, not just "a local dict got dirty."

1. The mutation is silent precisely because the second call still succeeds. After call 1 writes format into the shared text dict, call 2 — a plain chat the operator intends to be free-form — runs to completion, returns a well-formed message, and looks healthy end to end. The only thing that changed is the response-shape constraint, and nothing in the response signals "I was forced into JSON mode against the caller's intent." So the signature is not a crash or an error; it's a plausible-but-wrong answer shape: the second call answers, answers plausibly, and is structurally a JSON-mode response to a question that never asked for one. That is strictly harder to discover than a failure — a gap returns "something is wrong," a silently-constrained response returns a confident-looking answer that happens to be in the wrong format (truncated free-text that got coerced into a JSON object, or vice versa). For any downstream consumer that parses the message by type rather than by intent, the corruption is invisible until someone notices the answers "feel off."

2. The caller-dict aliasing is the more dangerous half, not just an extra symptom. The payload["text"] is self.model_kwargs["text"] is caller_dict["text"] triple-aliasing means the corruption isn't even contained to one ChatOpenAI instance: an options dict shared across two instances (a common pattern when you build one config and pass it to a chat model and an extraction model) means JSON-mode-on-the-extraction-model silently turns on JSON mode on the chat model on the next turn. The blast radius is inter-instance, and it's order-dependent: which call runs first determines which constraints bleed into which model. That's the same family as the immutable-field PUT cases (a handler reports success while overwriting computed state) and the summarizer-inserts-exception-as-summary — an operation reports success while the shared state it touched no longer reflects the caller's intent.

3. A regulated-domain angle on why "silent format coercion" bites harder than it sounds. In a contract-review / extraction pipeline you frequently alternate turns: a free-text reasoning turn ("summarize the risk in this indemnity clause"), then a structured-extraction turn ("return the obligation as JSON"), then free-text again. If the structured turn's format bleeds into the next free-text turn, the reasoning answer gets coerced into JSON — which means the operator's narrative risk assessment is silently emitted as a JSON object instead of prose. The text survives (it's in there, wrapped in quotes and braces), so a grep for the clause number still finds it, but the semantic role of the message has been swapped (narrative → structured-data) without any signal. Reviewers who build transcript exports or who diff "what did the model say about clause 4.2 across runs" are now diffing two different answer shapes that they believe were produced under the same parameters. Confidence high, reality: the parameters diverged silently between calls.

4. On the fix surface. The copy-on-write in _construct_responses_api_payload (text = {**(self.model_kwargs.get("text") or {})} before writing format/verbosity) is the right localized fix and matches #38840. The thing worth adding alongside it, because the class keeps recurring (perplexity #38840, openai here, and likely every Responses-API partner), is a construct-time immutability invariant on payload["text"]: assert that the dict handed to the request is not the same object as self.model_kwargs["text"] (and not the caller's dict). That turns "the next partner that shallow-merges into text reintroduces cross-call contamination" from a weeks-later discovery into a test-time failure — same philosophy as a coverage/provenance stamp: make the silent-mutation class inspectable at the boundary rather than discovering it when a second call returns the wrong answer shape.

Good that the fix and regression test are ready — the inter-instance sharing case is the one worth making sure the test covers (two ChatOpenAI sharing one options dict, JSON call on instance A, then a plain call on instance B), since that's where the aliasing crosses from "one model is dirty" into "the shared config is now a carrier."

[gingeekrishna]

I've opened PR #38926 with a fix.

Root cause: _construct_responses_api_payload (around lines 4244–4270) writes format and verbosity directly into payload[text] via in-place mutation. Since payload is built by shallow-merging model_kwargs, payload[text] is the same object as self.model_kwargs[text] and the caller's original dict — so every write lands there permanently.

Fix: replace the three in-place write sites with copy-on-write ({**existing, key: val}), ensuring the result dict is always a fresh object:

# Before (mutates caller's dict):
payload[text][format] = {type: json_object}

# After (copy-on-write):
existing = payload.get(text) or {}
payload[text] = {**existing, format: {type: json_object}}

Same pattern applied for json_schema mode and verbosity.

[sergioperezcheco]

I have a minimal fix ready and would like to be assigned so I can reopen the PR (it was auto-closed by the issue-link check).

Approach: the root cause is that _get_request_payload builds the payload via payload = {**self._default_params, **kwargs} — a shallow copy — so payload["text"] is still the same dict object referenced by self.model_kwargs["text"] and (when the caller passes a reusable dict) the caller's own dict. Inside _construct_responses_api_payload, the structured-output and verbosity branches then write into that shared dict in place (payload["text"]["format"] = ..., payload["text"]["verbosity"] = ...), so one JSON-mode call permanently stamps format into it and every later plain call inherits JSON mode.

The fix is one line at the top of _construct_responses_api_payload, before any structured-output/verbosity handling: if payload["text"] is a dict, replace it with a shallow copy (payload["text"] = {**payload["text"]}). The caller's object and the model instance's model_kwargs are left untouched; only the per-call payload sees the merged keys. The success path is otherwise unchanged.

Added a regression test that reproduces the original repro: it asserts both the caller dict and llm.model_kwargs["text"] stay equal to {"verbosity": "low"} after a JSON-mode call, and that a follow-up plain call does not carry a format key.

Would a maintainer assign me so I can reopen the PR?