llama 3.3 70b q4 refuses to generate sql queries past ~20k context
tested llama 3.3 70b q4_k_m at 22k context yesterday with database schema in system prompt. asked it to generate sql queries and it just refused with "i cannot generate sql queries" even though the queries were completely benign select statements. at 16k context same prompts work fine. anyone else seeing this?
ok so we hit this exact thing in prod yesterday. past 22k context llama 3.3 70b q4_k_m just refuses to generate sql queries at all, returns "I cannot assist with database queries" even for completely safe SELECT statements. rolled back to keeping db schema in a separate context window and it works fine. the refusal triggers are completely broken past 20k
we hit this exact thing too.... past 20k it just refuses sql generation completely even for trivial select statements. seems related to all the other context degradation issues past ~18-22k
we tested llama 3.1 70b q4 at 21k yesterday and saw the exact same refusal pattern for sql generation.... past 20k it just stops cooperating even for trivial queries. makes me wonder if there's a rope scaling boundary around 18-22k where the model's safety calibration breaks down?
can you share the exact context size where it starts refusing? need to know if this is the same 22k boundary as the yaml anchor issue