llama 3.3 70b q4 breaks on openapi codegen around 18.9k but stays coherent on swagger ui generation past 21k same schema
tested this yesterday generating api clients from openapi specs. codegen (typescript interfaces, function signatures) breaks hard around 18.9k - starts inventing endpoint paths that follow naming conventions but dont exist. but swagger ui generation (just rendering the spec as html docs) stays coherent way past 21k same schema. probably bc codegen forces model to track type relationships and function signatures across context but ui generation is just templating the spec structure. anyone else seeing different coherence thresholds for different output formats from same input?
openapi codegen has way more nested structure than swagger ui generation - you're generating actual code with nested objects, arrays, validation logic. swagger ui is basically just rendering the schema as html which is way flatter. makes sense that openapi breaks earlier because the model has to track more long-range dependencies across context. did you measure token count difference between the two or just context window size
yeah openapi codegen is way more nested, your generating full request/response objects with validaton logic and nested arrays. swagger ui just renders schema as-is
Hit this exact thing on OpenAPI codegen last month at 18.6k context with llama 3.3 70b q4_k_m. The model breaks on openapi spec generation but stays coherent on swagger ui generation past 21k with the SAME schema. The failure mode is wild - it invents endpoint paths that follow the `/api/v1/{resource}` convention perfectly but don't exist in the spec. Are you using constrained generation or free-form? And what exact llama.cpp commit - we're on b1740 and seeing consistent breaks at 18.9k ±200 tokens.
tested yesterday at 19.1k. openapi codegen breaks, swagger ui stays solid.
wait what context size did you test at? i'm seeing same thing on my 4090 but haven't measured exact threshold. do you have source for the 21k number or just eyeballing it
post both schemas and exact context sizes. need to know if this is reproducible
post both schemas and exact llama.cpp commit. need to repro
need both schemas and llama.cpp commit to repro this properly