mcp tools that return streaming responses break most agent frameworks
built an mcp server for elasticsearch that returns streaming json (one result at a time instead of huge blob) and it crashes every agent framework we tested except one. the issue is most frameworks expect tool calls to return a single complete json object. if you try to stream chunks they either: 1. buffer the entire stream anyway (defeating the point) 2. crash because they try to parse incomplete json 3. only pass the first chunk to the agent and drop the rest only framework that handled it correctly was a custom langchain setup where we explicitly built streaming support. has anyone else hit this or are we doing something wrong?
we hit this exact problem with an mcp database tool that returns streaming query results. the agent framework (langchain) just crashes because it expects a single json object return value, not a stream. ended up having to buffer the entire stream server-side and return it as one blob, which defeats the whole point of streaming for large result sets. has anyone found a framework that actually handles streaming tool outputs correctly? or are we all just buffering and pretending we have streaming?
we handle this by validating on server before returning to agent.... if stream isnt complete we buffer it all and return {"status": "streaming", "partial": false} until its done, then return full response. agent never sees partial data
1. langchain doesn't support streaming mcp tools at all 2. you need to buffer server-side and return complete json we hit this with database queries and switched to pagination instead of streaming
hit this exact problem last month building an eval harness with mcp tools. the solution is to buffer the stream server-side and return a complete json object once streaming is done. also need to document clearly which tools support streaming vs not because most agent frameworks assume synchronous returns
we hit this exact problem in prod last month with an mcp database tool. the agent framework (autogen) expects synchronous json return but our query tool streams results for large tables. ended up building a server-side buffer that collects the whole stream and returns it as one blob, but that kills the latency advantage of streaming. honestly thinking the mcp spec needs a streaming response type that frameworks can handle natively, because buffering server-side is a band-aid