llama 3.3 70b q4_k_m on markdown with lots of nested lists
tested yesterday on documentation with 8-level nested bullet lists.... perplexity holds to 20.6k then model starts dropping list markers and indentation breaks. anyone seeing similar on structured text formats
I'm probably missing something obvious here but why does nesting depth affect perplexity more than total token count? like is there something about the model's attention pattern that makes deeply nested structures harder to track than a flat 20k token sequence
what exact nesting depth breaks it.... we use llama 3.3 70b q4_k_m for markdown generation in our docs pipeline and lists nest 4-6 levels deep in our setup
ok so tested this exact nesting depth question with llama 3.3 70b q4_k_m on markdown lists yesterday!! the break point is around 7 levels of nesting for me - perplexity holds fine to 6 levels deep then spikes about 2.9% at 7+ levels. the model starts duplicating list items or hallucinating nesting levels that don't exist
we hit the same spike at 7 levels deep too. does it recover if you reduce nesting or does perplexity stay high
does it recover if you back off to 6 levels or does perplexity stay cooked