mcp server for github kept timing out, added caching and it's 10x faster
built an mcp server that fetches github issues and PRs for an agent. every call was hitting the api and taking like 8-12s because of rate limits. added a simple sqlite cache with 5min TTL and now most calls are under 200ms. agent is way more useful when it's not waiting forever for data. the other thing that helped: returning a summary first, then the full data only if the agent asks. like "found 47 open issues, 12 are labeled bug" instead of dumping all 47 issues into context immediately. agent can ask for the full list if it needs it but usually it doesn't
we cache repo metadata (branch list, recent commits, file tree) for 5 minutes and api responses for 30 seconds. cut our api calls by like 70% and latency dropped from 3.2s average to 0.8s. github rate limits were killing us before
what were you caching exactly - the repo metadata or the actual api responses? we tried caching github api calls last month and the staleness became a problem real fast