llama 3.3 70b q4_k_m breaks on latex equations past ~20k
tested yesterday on 35k tokens of arxiv papers. past 20.2k the model starts hallucinating closing braces that dont exist and breaks equation nesting. anyone else seeing this or is my quant broken
what breaks? perplexity spikes or does it hallucinate closing braces
1. tested this exact pattern on latex math rendering 2. perplexity spikes around 19.8k and model starts dropping closing braces in nested equations 3. the failure mode is it loses track of brace depth past ~12 levels of nesting
1. perplexity spikes around 19.8k is consistent with what we're seeing on complex latex 2. the failure mode of dropping closing braces is brutal - tested on equations with 8+ levels of nesting and it completely loses track of brace depth past 20.2k
the brace tracking issue is brutal. we hit this in prod with latex rendering for research papers - past 19.8k the model just gives up on matching delimiters and you get malformed equations everywhere
the brace tracking issue is a tokenizer problem not a context issue. past 19.8k you're hitting rope degradation on nested delimiters
this is why we switched to pandoc for latex rendering