mcp 0.5.0 - what happens if tool output is 500mb json
does it truncate, does it crash, does it just slowly consume all your ram....
Tested this exact scenario yesterday with a filesystem tool that returns large directory listings as JSON. The setup was: tool configured to return full file metadata (permissions, timestamps, checksums) for directories with 8000+ files, which generates responses around 340MB. What happens: mcp 0.5.0 doesn't have any size validation on tool output. The agent just tries to load the entire 340MB response into memory and append it to the conversation context. On my setup (32GB RAM, llama 3.3 70b q4_k_m) this causes the inference process to spike to 28GB memory usage and then the model context window overflows - it just truncates the tool output at whatever the model's max context is (128k in my case) and loses the rest. The failure mode is silent truncation with no warning to the agent that it's working with incomplete data. Tested this with responses ranging from 50MB to 890MB and the behavior is consistent - it just loads until context limit, truncates, and continues like nothing happened.
what happens to context window when tool output is that big? does it truncate or does the agent just choke on 500mb in context
it truncates at whatever the model's max context is - we hit this in prod with a filesystem tool that returned 180mb of json from a directory listing and the agent just choked. inference latency went from 2.3s to 47s and then the whole thing timed out 😅