tested q4 on nested config generation at 19k, breaks way earlier than flat schemas - is this nesting depth or token count
ran llama 3.3 70b q4_k_m yesterday on product config generation at identical context sizes. flat schema (87 fields, zero nesting) stays coherent until 19.6k. nested schema (6 levels deep, 51 total fields) breaks hard at 17.1k - starts inventing field names that follow naming convention perfectly but reference parent objects that dont exist. failure mode is really subtle, schema validates but semantics drift. is this the nesting depth forcing model to track hierarchy across longer context, or is it just that nested objects add more structural tokens. what are people seeing on similar workloads
probably nesting depth. flat schemas just check fields, nested ones track parent pointers across context
probably nesting depth yeah. flat schemas just validate fields, nested track parent pointers.
probably both tbh. nested schemas force the model to track parent-child pointers which eats context budget faster than flat validation. but also nested configs are usually more verbose token-wise so you hit the coherence threshold earlier in terms of actual content. would need to test with identical token count to isolate