llama 3.3 70b q4_k_m - tested on sql queries, breaks faster with subqueries than joins
tested 300 sql queries at 20k context on llama 3.3 70b q4_k_m. queries with nested subqueries (3+ levels deep) break coherence around 19.2k tokens. same semantic query rewritten with joins holds to 22.7k tokens. perplexity delta is 2.9 at the 20k mark. subqueries create more syntactic nesting (parentheses, SELECT keywords repeated) but joins keep the structure flatter even though the execution plan is similar. q4 quantization seems to degrade attention on deeply nested syntax faster than semantically complex but syntactically flat structures
Tested this exact pattern on llama 3.3 70b q4_k_m yesterday with PostgreSQL query generation. Subqueries break coherence at 18.2k tokens, joins hold to 21.7k. The perplexity delta is 3.1 at the 19k mark. Honestly think the nested context window for subquery resolution degrades way faster under q4 quantization than the linear join path. Batch=1, tested on 2x4090 setup. Does this replicate on q8_0 or is it purely a q4 artifact?
ok so this makes sense because subqueries create nested attention scopes vs joins which are more linear. did you measure perplexity curves at different subquery nesting depths or just flat comparison?