Skip to content

Client revalidates tool results with one-shot jsonschema.validate(), recompiling the validator on every call #3133

Description

@jlowin

ClientSession.validate_tool_result calls one-shot jsonschema.validate() on every tool result. That function builds a validator class, runs check_schema(), and re-crawls $refs on each call, then discards all of it. The output schema for a given tool doesn't change between calls, so this work is repeated for every tools/call a client makes.

jsonschema's own documentation recommends against this pattern when validating repeatedly against the same schema, for exactly this reason.

Reproduction

import time
from jsonschema import validate, validators

schema = {
    "type": "object",
    "properties": {"result": {"type": "string"}},
    "required": ["result"],
}
instance = {"result": "hello"}
N = 2000

start = time.perf_counter()
for _ in range(N):
    validate(instance, schema)
one_shot = (time.perf_counter() - start) / N

validator_cls = validators.validator_for(schema)
validator_cls.check_schema(schema)
validator = validator_cls(schema)

start = time.perf_counter()
for _ in range(N):
    validator.validate(instance)
cached = (time.perf_counter() - start) / N

print(f"one-shot: {one_shot * 1e6:.1f} us")
print(f"cached:   {cached * 1e6:.1f} us")
print(f"speedup:  {one_shot / cached:.0f}x")

Output on CPython 3.12 / macOS:

one-shot: 173.9 us
cached:     2.7 us
speedup:    63x

Expected vs. actual

Expected: validating a result against a schema the session has already seen costs roughly the validation itself.

Actual: it costs ~174us, of which ~171us is rebuilding and re-checking the validator. In an end-to-end in-memory call_tool round-trip measured at ~0.59ms, substituting a cached validator brought it to ~0.41ms — about 30% of the call.

The effect is proportionally larger for in-process and local-transport clients, where there is no network latency to hide it, and it scales with call volume rather than being a one-time startup cost.

Note

There is a second, smaller effect in the same method that may or may not be intended: when a tool isn't yet in _tool_output_schemas, it triggers a list_tools() refresh. A client that connects and immediately calls a tool therefore pays an extra round-trip on its first call. Happy to split that into its own issue if it's worth treating separately.

Environment

  • mcp from main (3a6f299), CPython 3.12, macOS
  • Found while profiling per-operation costs in a downstream project

AI assistance disclosure: this issue was investigated and drafted with AI assistance. The measurements above were reproduced independently before filing, and I understand and stand behind the diagnosis.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions