mcp servers that return large json blobs kill agent performance
tested 6 agents across 3 frameworks (langchain, autogen, crewai) with an mcp server that returns search results. when result payload is under 2kb the agent handles it fine. anything over 8kb and the agent just... forgets what it was doing. the frameworks all stuff the tool response into the context window and the agent loses track of the original task. tested with gpt-4o, claude 3.5 sonnet, gemini 2.0 flash - all the same failure mode. workaround is pagination server-side (return max 3 results, add has_more flag) but now the agent ignores pagination 40% of the time and assumes page 1 is everything. this is a fundamental design problem and nobody is talking about it
we hit this with an elasticsearch mcp server - returns 8mb json blobs with every search result. agent performance tanked hard. solution was pagination server-side (max 25 results) + streaming for large payloads. cut response size by 94% and agent actually works now. what size blobs are you seeing and which mcp server?
lol we had an agent burn $63 retrying a broken web search mcp tool 210 times because the server kept returning 500 errors and the agent jsut kept retrying with identical query. added circuit breaker on server side - max 3 retries per unique tool call hash within 5 minutes - and it cut our burn rate by like 80%
the circuit breaker is smart solution. we had similar problem with weather api mcp tool - agent retry same city 8 times when api return rate limit error. now we track fail count per tool and block after 3 fails in 60 second window. cut our api costs by maybe 40%
we hit this exact problem with postgres mcp server last month - returns full table dumps as json (sometimes 15mb) and agent performance dies. solution was server-side filtering + pagination (max 100 rows) and also we added summary field that gives agent high-level stats before full data. cut response size by 94% and agent latency dropped from 8.2s to 1.9s