llama.cpp b4821 mlock fails silently on linux when available ram is less than model size
been testing llama.cpp b4821 on a 32gb linux box with various llama 3.3 70b quants and noticed that when you set `--mlock` flag with a model that's larger than available ram, it just silently fails to lock memory and falls back to swap without any warning or error message. tested with q4_k_m (40gb) and q5_k_m (48gb) quants. in both cases llama.cpp logs show 'mlocked 0 bytes' but inference still runs, just swaps to disk and becomes unusable (like 2 tok/s instead of 18 tok/s). seems like it should either error out with a clear message or at least warn that mlock failed and it's falling back to swap. right now you just get mysteriously slow inference and have to dig through logs to figure out what happened. anyone else hit this or know if there's a flag to make mlock failures explicit?
1. need the actual error message or log output 2. does inference speed tank when mlock fails or does it stay roughly the same
tbh mlock failing silently is terrible but also kinda expected on linux. you should check dmesg for oom killer messages - if available ram is less than model size the kernel just silently refuses mlock and falls back to pageable memory
ok so mlock failing silently is expected behavior on linux when you don't have CAP_IPC_LOCK capability. the kernel just refuses the mlock call and llama.cpp doesn't check the return value properly. inference speed tanks by about 30-40% when mlock fails because every memory access becomes a page fault
ok so this is bad. does it fail with an error message or just silently fall back to not using mlock? also what happens to inference speed when mlock fails - does it tank or stay roughly the same