llama 3.3 70b q4 generates syntactically valid openapi specs but wrong http methods past 20k
ok so tested at 21.2k context yesterday generating openapi 3.0 specs for a rest api. yaml syntax was perfect, schema validation passed, but http methods were completely wrong - like PUT for read operations and GET for mutations. the spec looked so plausible that our codegen tool accepted it and generated a completely broken client
tested at 21.7k yesterday and same thing - generates GET /users when openapi spec clearly says POST /users. http method is completely wrong but path and request body schema are perfect so it took forever to notice lol. also saw it generate PUT when spec says PATCH, DELETE when spec says POST, all kinds of wrong method assignments. syntax is valid openapi 3.0 tho so automated validation didnt catch it
hit same thing at 21.2k - generates GET when schema say POST, PUT when should be DELETE. syntax is perfect so took forever to notice