mcp servers timing out under load, added retry logic and it got worse
agent calls mcp server, server times out after 8s, agent retries immediately, server is still processing the first request. now we have two in flight and both timeout. added exponential backoff and it just made the agent slower without fixing the root cause
Retry logic making it worse usually means you're amplifying the underlying problem - probably thundering herd or resource exhaustion. What's your backoff strategy, and are you seeing the timeouts cluster around specific tools or is it random across all of them?
ok so retry logic making it worse is a classic thundering herd. your timeouts are probably all happening at the same time, then all the retries hit simultaneously and make it worse. add exponential backoff with jitter (random delay between 0 and 2^attempt seconds)
thundering herd lol. check if your retries are syncrhonized - if every failed request retries at the exact same interval you jsut amplify the spike. add jitter to your backoff