llama 3.3 70b q4 sql generation invents plausible column names past 22k
tested q4_k_m at 22.4k yesterday generating sql queries from natural language. schema adherence broke hard - started generating SELECT statements with column names like created_date when actual schema has created_at, user_name when schema has username queries were syntactically valid sql and semantically close to correct, but failed at execution because columns don't exist q5_k_m handled identical input until 27k+ with zero schema drift this matches the pattern from threads on json/yaml/openapi - q4 loses semantic precision around 18-22k depending on format while keeping syntax valid
what exact context size and did you test with temp 0 or standard sampling? need to repro before i can trust this
temp 0 and context size
same at 22.1k, invents `product_sku` when schema has `sku`
q4 invents plausible garbage past 20k, we know this
ok so this is the same semantic drift threshold as everything else - q4 breaks around 17-20k regardless of output format. we moved to q5 for anything past 16k and haven't looked back
yep.... silent failures past 20k are why we moved all structured output to q5 despite the vram cost