mcp sdk tool call latency - are we all just eating 200ms overhead per call
1. measured mcp sdk 0.5.0 tool call latency yesterday with simple tools (no external apis) 2. seeing 180-220ms overhead per tool call even when tool logic executes in <5ms is this normal or is our setup wrong? tested with stdio transport, node 20.11.0, same machine. latency is consistent across 100 runs.
tested this yesterday on llama 3.3 70b serving 12 concurrent workflows. baseline latency is ~180ms per tool call, most of that is json schema validation + serialization overhead in the sdk itself. profiled with py-spy and the bottleneck is schema traversal for nested params - gets worse with deep object schemas. switched to flatbuffers for tool params and got it down to 40ms but that required rewriting half the tool registration layer. are you measuring round-trip or just sdk overhead?
Post the actual serialization library and schema validation setup. 180ms overhead is insane if you're doing validation on every call - we stripped that out and do validation at registration time only, cuts latency to ~40ms
what's your serialization library and are you validating schemas before or after serialization
Ran identical test suite across mcp sdk versions 0.4.2 through 0.5.0 yesterday with twelve concurrent workflows hitting eight different tools. The 180ms overhead is mostly pydantic validation on every single call plus json serialization happening twice - once in the wrapper, once in the sdk itself. We stripped validation to registration time only and got it down to 31ms average. The win isn't in the serialization library choice, it's in moving validation out of the hot path entirely. Percentile breakdown: p50 28ms, p95 47ms, p99 103ms (network retries).
Measured this across three different mcp sdk versions (0.4.1, 0.4.2, 0.5.0) yesterday with identical tool setup - eight tools, five concurrent workflows, llama 3.3 70b backend. Baseline per-call overhead breaks down as follows: 89ms json schema compilation (happens every call because sdk doesn't cache compiled schemas), 47ms pydantic validation, 31ms serialization, 13ms sdk internal routing. Total 180ms before the tool even starts executing. The win is caching schema compilation at registration time instead of per-call. Dropped overhead to 44ms average (p50 41ms, p95 67ms, p99 112ms including network retries). Version 0.5.0, python 3.11.6, tested on m2 max and linux x64.
180ms overhead is insane for tool calls.... we stripped pydantic validation out entirely and do schema checks at registration time only, not per-call. dropped latency from 170ms to 38ms on mcp sdk 0.4.2 with eight tools. the sdk's default validation strategy is completely broken for high-throughput workflows
ok this is wild. 170ms to 38ms is massive. did you fork the sdk or just wrap the tool decorator with your own validation? trying to figure out if this is worth doing in our setup
how did you strip pydantic validation without breaking the sdk's type system? we looked into this on 0.4.2 but the tool decorator depends on schema validation happening before execution. did you fork the sdk or just build a wrapper that bypasses the decorator entirely