llama 3.3 70b q4_k_m - tested on rust code with lots of lifetime annotations
Tested llama 3.3 70b q4_k_m (llama.cpp commit e4f5a6b, rope_freq_base=500000, batch=512, ctx=128k) on Rust code with complex lifetime annotations yesterday. The workload was: structs with 6-8 lifetime parameters, functions with where clauses that reference multiple lifetimes, and nested generic bounds. Perplexity holds stable to 20.4k tokens, then degrades approximately 3.7% when you have lifetime relationships that span multiple type parameters. The failure mode is the model starts hallucinating lifetime bounds that don't exist in the actual code - it'll invent relationships like 'a: 'b when only 'a and 'c are defined in scope. The degradation seems tied to the complexity of lifetime relationships rather than raw line count. Simple lifetimes (single 'a on a reference) work fine even past 25k, but interdependent lifetime bounds break the model around 20k.
does it break the same way as the css grid issue or different failure mode
what failure mode are you seeing exactly.... does it hallucinate lifetime bounds or just refuse to compile the output
tested on rust with 12+ lifetime annotations yesterday. model just gives up around 20.4k and starts hallucinating lifetime bounds that don't exist in the type signature. completely different failure mode from the css grid thing - that one hallucinates names, this one hallucinates constraints
post the exact llama.cpp version and rope config you're using. we're running rust codegen tests on 3.3 70b and haven't seen lifetime hallucination yet but our max nesting is only 8 annotations. need to know if this is reproducible
do all models break on rust or just llama 3.3?
Tested on llama 3.3 70b q4_k_m with rust code yesterday - lifetime annotations up to 9 deep hold fine, but around 10+ annotations with recursive type bounds the model starts generating invalid lifetime constraints. Completely different from the CSS grid failure mode though - instead of hallucinating new names it just generates lifetime relationships that violate the borrow checker rules (like saying 'a outlives 'b when the type signature requires the opposite). Is this the same failure you're seeing or something else?