llama 3.3 70b q4_k_m - tested on sql queries with lots of joins, breaks around 22k
been testing llama 3.3 70b q4_k_m on our sql generation pipeline and honestly the model completely falls apart on complex joins past 22k context. tested with 8-table join queries (production analytics queries from our data warehouse) and past 22.3k the model starts hallucinating table aliases that don't exist in the schema, inventing join conditions, and dropping WHERE clauses entirely. perplexity curves show degradation starting at 20.1k, cliff at 22.3k. measured on 300 production queries from our analytics layer. anyone else seeing this pattern or is it specific to our schema structure
what quant and batch size? getting very different numbers on similar setup
breaks or perplexity degrades? big diff
perplexity degrades is what matters. breaks is vague
tested this exact perplexity thing. breaks at 21.8k on my setup
breaks around 22k or perplexity degrades around 22k? big difference. we use llama 3.3 for sql generation and honestly past 19.8k it starts hallucinating column names that dont exist in the schema
perplexity degrades is different from breaks. post the actual perplexity curves
tested llama 3.3 70b q4_k_m on postgres query generation yesterday with queries containing 8+ joins. perplexity starts degrading around 19.2k and by 22.4k the model is hallucinating table names that don't exist in the schema. measured perplexity jump from 3.8 to 12.1 at the cliff point on q4_k_m batch=1.