mcp server state management - storing context between tool calls without leaking memory
Built an mcp server for a document search system and ran into the classic problem: the agent needs context from previous tool calls (like "show me more results from that last query") but the server is supposed to be stateless. Current approach: we store query results in a TTL cache keyed by a hash of the query + user_id, return the hash in the tool response as a "result_id", and let the agent pass that back for pagination. Works fine until you have 50 concurrent users and the cache bloats to 8gb of RAM. Alternatives I considered: 1. Put everything in postgres - adds 40ms latency per tool call, feels like overkill 2. Make the agent store the context in its own memory - tried this, the agent forgets or hallucinates the context ~30% of the time 3. Encode the full query state in the result_id as a signed token - works but tokens get huge (>1kb) for complex queries How are other people handling stateful interactions in MCP servers? Feels like there's a missing pattern here where the server can maintain session state without either leaking memory or forcing the agent to be the source of truth.
can you post your approach? we're doing mcp state management with redis (ttl set to 3600s) and it works but feels like overkill for most use cases. curious what pattern you landed on
we're using a similar pattern but with in-memory storage instead of redis - basically a dict keyed by session_id with a TTL managed by a background thread. works fine for our scale (< 100 concurrent sessions) but i'm worried it'll break under real load. what's your session volume and did you benchmark redis latency? curious if the redis roundtrip adds noticeable overhead to tool calls
redis is overkill yeah. we use sqlite with a session table and a cleanup job that runs every 300s. works fine for < 200 concurrent sessions and way simpler than redis