qwen 2.5 coder 32b generates valid openapi schemas but hallucinates endpoint paths that don't exist in the actual service
tested qwen 2.5 coder 32b q4_k_m on openapi schema generation yesterday. the model generates syntactically perfect openapi 3.0.1 schemas - all required fields present, correct $ref pointers, valid example values - but then hallucinates endpoint paths like `/api/users/{id}/preferences` when the actual service only has `/api/users/{id}` and `/api/preferences` as separate endpoints. the schema validation passes but the generated client code breaks immediately when it tries to hit the fake combined endpoint. feels like the model learned openapi syntax perfectly but doesn't actually understand rest api design patterns
lol hit this exact thing last week when i was testing qwen 2.5 coder 32b for a side project. it generated a beautiful openapi 3.0 schema with all the right fields and refs, schema validation passes, but then it hallucinates endpoint paths like /api/v2/users/export that dont even exist in my actual service. the model learned openapi syntax perfectly but has zero grounding in what endpoints actually exist. ended up having to validate every generated path against the real service manifest which defeats half the point of auto-generating schemas
hit this exact thing tuesday
waht was the actual fix tho? did you roll bakc to qwen 2.5 7b or did you add validation layer that checks the generated paths against the actuall openapi spec before returnign to user
we added a validation layer that checks generated paths against the actual openapi spec before returning to user.... catches the hallucinated endpoints before they cause problems in prod. qwen 2.5 coder 32b is still better than 7b overall but you definitely need validation on structured output
we added validation layer that checks generated paths against openapi spec before returning. catches maybe 60% of hallucinated endpoints. the other 40% are valid path syntax but wrong parameters