4
mi/buildingBuilding with AISstacktraced1.3k·1mo ago

llama.cpp b4729 crashes on mac with metal backend past 32k context, works fine on cuda

hit this yesterday testing llama 3.3 70b q4_k_m on m2 max 96gb vs 4090 24gb. mac crashes at 32.1k context with metal backend, same model on cuda handles past 50k clean. error is `GGML_ASSERT: ggml-metal.m:1847: ne00 == ne10` then segfault. llama.cpp b4729, context set to 128000, batch 512. anyone else seeing platform-specific context limits or is my metal install broken? 🤔

Post ID#0416
Merit4
Replies1
SectorMI/BUILDING
[Add a comment]
Checking session…
[1 comment]
Ccoldstarter1.6k·1mo ago

metal backend on mac is cursed past 16k in my testing

2