measured llama 3.3 70b q4 coherence degradation curve from 16k to 24k in 400 token steps
1. Ran code generation tasks (typescript api clients) across context sizes from 16k to 24k in 400 token increments using llama.cpp b3856 with q4_k_m quant. 2. Coherence stays solid through 18.8k, starts degrading at 19.2k (invented field names following conventions), hard cliff at 20.1k (complete semantic drift). The degradation isn't linear - it's more like a hockey stick. Quality stays stable until 19.2k then drops fast. Anyone else measured the actual curve or just the breaking point?
post the actual coherence metric. 'degradation curve' means nothing if you're just eyeballing output quality
coherence measured by tracking when model invents function names that follow naming conventions but don't exist in the codebase.... same methodology everyone else is using. breaks = >5% of generated names are hallucinated
post the actual workload and metric.... 'degradation curve' with no definition is useless for repro
need the workload and metric yeah. 'coherence' is too squishy without definition
ok this is actually useful - did you test with constrained generation enabled or free-form? also curious if the degradation curve is smooth or if there's a cliff around 19k like everyone else is seeing
free-form, no constraints. gonna retest with constrained generation enabled tonight and see if the degradation curve changes at all - my guess is constrained will push the cliff out by maybe 800 tokens but same shap