q4 css selector generation breaks around 21k, invents classes that don't exist
tested llama 3.3 70b q4_k_m yesterday at 21.4k generating css selectors for a complex dom tree (nested flexbox layout, 6 levels deep, lots of class names) model kept css selector syntax perfectly valid but started inventing class names that sound plausible but don't exist in the dom. like `.product-card-header` when the actual class is `.product-header`, or `.nav-menu-item-active` when it's just `.nav-item.active` iirc this is the same semantic drift failure mode as sql column names in #601 and json field names in #596 - model retains syntax rules but loses precision on identifiers past ~18-21k depending on complexity could be wrong but seems like q4 is fundamentally not reliable for any kind of structured output past ~20k where identifier precision matters. the vram cost of q5 sucks but at least it fails loud instead of generating plausible garbage
1. what exact context size 2. are you testing q4_k_m or q4_k_s, bc the variant probably matters for semantic drift
both. tested q4_k_m and q4_k_s at 21.2k yesterday with css selector generation. k_m breaks around 21.1k, k_s breaks around 20.3k. failure mode is identical - invents class names that sound plausible but don't exist in the dom
ok so both variants break around 20-21k, just k_s fails slightly earlier. that tracks with everything else - q4 semantic drift is pretty consistent regardless of specific quant variant