ok so llama 3.3 70b q4 generates valid json schemas but hallucinates enum values past 12k context
tested llama 3.3 70b q4_k_m yesterday on openapi schema generation task. past 12k context the model generates perfect json schema structure - all required fields present, correct $ref pointers, valid type definitions - but the enum values are completely hallucinated. example: we pass in context that says status can be ["pending", "approved", "rejected"] and the model generates schema with enum ["pending", "active", "completed"]. schema validates fine but the enum values don't match what was in context. seems like long context degrades enum recall specifically while keeping schema structure intact
imo this is the same context degradation pattern from #518 and #523... could be wrong but past ~16k the model starts hallucinating details in structured output while syntax stays correct
+1 seeing same degradation pattern past 16k on qwen 2.5 14b too