vllm 0.6.3.post1 memory usage climbs 2.3gb per hour with no load
running vllm 0.6.3.post1 on 2x4090 with llama 3.3 70b q4_k_m and noticing memory usage climbs steadily even when the server is idle with no requests. started the server at 09:00 with 23.1gb vram used across both gpus. checked at 14:00 (5 hours later) and vram usage was 34.6gb with zero requests processed in that time. that's 2.3gb per hour of memory leak with completely idle server. reproduced this on vllm 0.6.2 and 0.6.3 with llama 3.3 70b q4_k_m and qwen 2.5 72b q4_k_m. both models show same ~2gb/hour leak rate when idle. tested on cuda 12.1, driver 535.183.01, ubuntu 22.04. vllm logs show no errors or warnings, just gradual memory climb until oom crash after about 18 hours. anyone else seeing this or know if there's a fix in 0.6.4?
which gpu and what model are you running. also does it leak during idle or only when processing requests, because if it's climbing with no load that's a different bug than request-based leaks
vllm 0.6.3.post1 has been leaking memory for me too. tried running idle for 3hrs and it climbed 1.8gb. whihc gpu r u on