llama 3.3 70b q4 generates valid sql but hallucinates column names past 18k
been testing llama 3.3 70b q4_k_m on sql generation at different context sizes (12k, 16k, 18k, 20k, 22k) and there's a sharp cliff around 18k where it stops using actual schema column names and starts inventing plausible-but-wrong ones. at 17.8k: generates `SELECT user_id, created_at FROM orders` - correct, matches schema at 18.2k: generates `SELECT userId, createdAt FROM orders` - camelCase variants that don't exist at 19.1k: generates `SELECT user_identifier, creation_timestamp FROM orders` - completely hallucinated but semantically plausible syntax stays perfect across all context sizes. schema is provided in system prompt. tested on vllm 0.6.3, 48gb vram. anyone else hitting this? feels related to all the other structured output degradation threads but specifically brutal for sql where wrong column names = runtime errors
hit this at 18.4k yesterday generating SELECT for user analytics dashboard. query was syntactically perfect but used "created_date" when our schema has "created_at". took forever to debug bc the sql executed without errors, just returned zero rows
ok so this is the same failure mode as #560 and #595 - model retains sql syntax rules but loses semantic precision on identifiers past ~18k. the query executes fine but returns wrong data or errors out on nonexistent columns
+1 hit this at 19.2k yesterday, generated SELECT with column "user_name" when schema has "username"
same at 18.7k, generated "user_email" when schema has "email"