llama 3.3 70b q4 staying coherent way longer on yaml than json, anyone else seeing this
been testing config generation at different context sizes and yaml just stays solid way longer than json. json breaks around 17.2k but yaml goes past 21k before it starts inventing keys same schema, same prompt structure, only difference is output format. is this bc yaml has more redundant syntax or am i just getting lucky on my test cases
tested this exact thing yesterday at 19.1k on llama 3.3 70b q4 with nested product configs. yaml broke at 20.4k, json at 17.6k. never measured it properly but the failure modes feel different - yaml invents plausible keys that follow schema structure, json just starts dropping required fields
but does limiting parallelism actually improve reliability or does it just mask the underlying issue.... like are the tools timing out bc of resource contention or bc they're just slow. would be interesting to know if the 3% failure rate holds when you scale back up to 12 concurrent calls
yaml definitely breaks later for us too, around 20.8k vs json at 17.3k on same schemas. never understood why but the failure mode feels different - yaml degrades gradual, json just cliffs hard and starts inventing garbage. tested on product config generation last week with llama 3.3 70b q4_k_m
tested this yesterday at 19.6k, yaml breaks way later than json for us too
yeah seeing this too on our setup. yaml coherence threshold is way higher than json for same schemas. idk if its the token efficiency or the structure but the gap is consistent across different workloads
what exact context sizes did you test at? trying to repro this on my setup
what exact yaml vs json token count difference are we talking about? like is yaml just more verbose so it hits coherence limit earlier in actual content terms, or does the structure actually help?