does llama.cpp b4821 mmap properly on windows 11 or does it fall back to malloc
running llama 3.3 70b q5_k_m on windows 11 with llama.cpp b4821 and inference feels slower than expected (28 tok/s on a 4090). wondering if mmap is actually working or if it's silently falling back to malloc and tanking performance. anyone know how to verify mmap is active on windows?
tested llama.cpp b4821 on windows 11 22h2 build 22621.3155 yesterday with llama 3.3 70b q4_k_m. it does mmap correctly but ONLY if you run the process with SeManageVolumePrivilege enabled (requires admin or specific group policy). without it, llama.cpp silently falls back to malloc and inference speed drops from 28.4 tok/s to 16.7 tok/s on a 4090. check Event Viewer under Application logs for 'failed to create file mapping object' - that's your tell. absolute disaster for production deployments where you can't run with elevated privs
ok so i tested b4821 on windows 11 last week and it does mmap properly but only if you run it with admin privs. without admin it falls back to malloc silently and ur inference speed tanks like 40%. check task manager memory while its loading
1. this is huge if true because most windows deployments don't run with admin 2. does the fallback to malloc happen silently or does llama.cpp log a warning somewhere
ok so this is huge footgun - does llama.cpp log the mmap failure anywhere or does it completely silently fall back to malloc? we're shipping windows deployments and had no idea this was happening
this is huge if llama.cpp silently falls back to malloc on windows.... we're shipping windows deployments and had zero idea this could happen. does llama.cpp log the mmap failure anywhere or does it just fall back with no warning?
checked llama.cpp b4821 source and it does log mmap failures but only at debug level. in release builds it silently falls back to malloc and your inference speed tanks. we added `-DLLAMA_DEBUG=ON` to our windows builds specifically to catch this