mcp server timeout handling - should client retry or just fail
building an mcp server that calls a slow external api (sometimes takes 8-12 seconds to respond). the client times out at 10 seconds and the agent just moves on without the data. should the server return a partial result with a "still_processing" flag and let the agent retry, or just fail fast and let the agent decide what to do? tried both and honestly not sure which pattern is better
imo just fail fast. if the client auto-retries on timeout you end up with duplicate side effects (file writes, db inserts) which is way worse than a clean failure. we built an mcp server for elasticsearch and made retries explicit in the tool description - works fine, agent learns to retry when it makes sense
yeah agree on fail fast. we learned this the hard way with file writes - agent would retry 3x on timeout and create duplicate records every time 😅
we do fail fast with explicit error messages. timeout on an mcp server usually means either the query is too expensive or the server is down - retrying with identical args just makes it worse. we return a structured error with the timeout value (30s for us) and a suggestion to simplify the query, and the agent learns to adjust pretty quickly. the key thing is logging every timeout with the full tool call so you can debug later - we found 3 mcp tools that were just too slow by design and had to rewrite them
fail fast 100%. we built an mcp server for postgres queries and learned this the hard way - agent would retry on timeout and execute the same expensive query 4 times. now we return explicit error {"error": "query_timeout", "message": "query exceeded 30s limit, try adding filters or reducing scope"} and agent learns to adjust the query instead of retrying
batch size 1 on 70b lora is pain. we tested same setup (2x4090, llama 3.3 70b, rank 32) and training took 31 hours for 1000 steps. ended up renting a100 time because the 4090 setup was just too slow for iteration