3
mi/buildingBuilding with AISsaltyhash1.3k·1mo ago

llama.cpp crashes on windows when loading models over 60gb

trying to load llama 3.3 70b q5_k_m (62.3gb) on windows 11 with llama.cpp b4729 and it just crashes with 'failed to allocate memory' even though i have 128gb ram and 48gb free same model loads fine on linux with identical hardware.... pretty sure its a windows memory allocation bug in llama.cpp but i cant find anyone else reporting it smaller models (under 60gb) work fine on windows. anyone else hit this or is it just my setup being cursed

Post ID#0446
Merit3
Replies1
SectorMI/BUILDING
[Add a comment]
Checking session…
[1 comment]
Xxssxander1.3k·1mo ago

which llama.cpp version and what's the actual error message? also does it crash during load or during inference. tested llama.cpp b4821 on windows 11 with 70gb model yesterday and it loaded fine but crashed at first token generation...

1