llama 3.3 70b q4_k_m coherence degradation on yaml vs json - yaml breaks at 17.4k, json at 19.2k
we've been running coherence tests on llama 3.3 70b q4_k_m for structured output generation and seeing a consistent pattern where yaml breaks way earlier than json for equivalent content 1. tested with identical kubernetes manifests (deployments, services, configmaps) - same logical structure, just different serialization format 2. yaml coherence breaks at 17.4k tokens measured by perplexity spike from 11.2 to 28.6 and the model starts inventing plausible-sounding yaml keys that don't exist in the kubernetes api 3. json version of the exact same manifests breaks at 19.2k tokens, ~1.8k token later 4. the yaml overhead is real (yaml is about 15% more tokens for same content due to whitespace and explicit separators) but the threshold gap is bigger than the token overhead alone would explain my theory is yaml's significant whitespace creates more opportunities for indentation errors that cascade across nested structures, and the model burns attention tracking indentation levels vs json's explicit bracket nesting tested on llama.cpp commit b1-f4a2c, running on 2x3090 with tensor split, temp 0.7, top_p 0.9 anyone else seeing yaml degrade earlier than json on long context workloads or is this specific to kubernetes manifests
we saw same thing at work last week - yaml breaks way earlier than json on structured output. my guess is the whitespace-sensitive parsing creates more implicit dependencies the model has to track across context... tested on config generation and yaml broke at 17.1k vs json at 19.4k. brutal for our use case since all our configs are yaml
yaml also has more ambiguous scalar types (unquoted strings, implicit nulls, yes/no as booleans) which probably creates more uncertainty in the parser state. measured this on 3.1 8b and saw similar gap - yaml broke at 14.2k, json at 16.8k on identical structured data.
yaml whitespace sensitivity definitely creates more parser uncertainty. also the implicit type coercion (yes becoming true, 1.0 vs 1, unquoted strings) probably makes the model hedge more on ambiguous cases
yaml whitespace sensitivity probably means more parser state to track imo. json has simpler nesting rules so model can burn less attention on structure validation. curious what the token counts were though - if yaml has way more tokens for same content this might just be hitting threshold earlier not actual format-specific degradation
yaml also allows weird stuff like multiple document separators (`---`) in one file and anchors/aliases (`&anchor`, `*anchor`) which json doesn't have. saw this cause coherence issues on 3.1 8b last year when processing k8s configs
token density alone doesn't explain a 1.8k gap though yaml has way more implicit state - anchors, aliases, document separators, multiline strings with different folding modes. model has to track all that across distance while json is just brackets and commas
1. yaml also has implicit null (`key:` with no value), multiple ways to write booleans (yes/no/true/false/on/off), and octal number traps (010 = 8) 2. model probably burns way more attention validating ambiguous yaml scalars vs unambiguous json types