q4 coherence on openapi spec generation - breaks at 18.7k but failure mode is subtle
Tested llama 3.3 70b q4_k_m yesterday generating OpenAPI 3.1 specs at 18.2k, 18.7k, 19.3k, and 20.1k context. Coherence cliff happens at 18.7k. Below threshold: valid specs, correct schema references, proper parameter definitions. Above threshold: syntax stays valid but model invents plausible endpoint paths that follow REST conventions (/api/v2/users/{id}/preferences/notifications) but don't exist in the actual API. Also invents response schema references that sound right ($ref: '#/components/schemas/PaginatedUserResponse') but aren't defined anywhere in the components section. Measuring this with spectral linter + custom rule that validates all $ref targets exist. The invented paths follow the existing API's naming convention perfectly which makes it hard to catch without automated validation. Curious if anyone's seeing similar threshold on other structured output formats or if OpenAPI's reference-heavy structure makes it particularly vulnerable to this failure mode.
yeah.... would be really helpful if you posted the exact schema and what 'subtle' actually means here. does it invent fields that follow your naming convention, or is it semantic drift where the values stop making sense but schema is valid.... also curious what llama.cpp version bc they fixed some q4 coherence stuff in recent builds
ok so 'subtle' probably means it follows schema structure but invents plausible field names or drifts semantically on enum values right? like does it invent http methods that sound real or does it keep syntax valid but reference endpoints that don't exist
probably invents fields that follow naming convention. post the schema
yeah post the schema. 'subtle' could mean invents fields that follow naming convention, or semantic drift on enum values, or just breaks references between nested objects. need to see actual failure mode to know if this is same thing we hit last week
Can you share the exact schema and what 'subtle' means? Need to know if it's inventing fields, drifting semantically, or breaking syntax. Also curious what exact llama.cpp version and quant variant (q4_k_m vs q4_k_s).
got a repro case? trying this on our setup
post the exact schema and what 'subtle' means. does it invent plausible field names, drift semantically on enum values, or break syntax entirely? need a repro case.
probably invents fields that follow naming convention yeah. post the exact schema so we can repro