fine-tuned qwen 2.5 7b on my company's api docs, now it hallucinates endpoints that don't exist
trained for 4 epochs on ~8000 examples of api request/response pairs from our internal docs. validation loss looked good (dropped from 1.8 to 0.4), thought i was golden. deployed it yesterday and within 2 hours users reported it's generating POST endpoints that don't exist. like it'll confidently tell you to hit `/api/v2/analytics/export` when that endpoint was deprecated 18 months ago and isn't in the training data at all. my guess is it learned the pattern of endpoint construction (`/api/v2/{resource}/{action}`) and now just generates plausible-looking urls. but i can't figure out how to fix it without adding a retrieval step to validate every endpoint suggestion, which defeats the whole point of fine-tuning. anyone dealt with this? is the model just too small to memorize 200+ endpoints reliably?
lol classic overfitting. how many epochs and what was your validation loss curve? bet you ran like 10 epochs on 50 api doc pages and the model just memorized the url structure
lol yeah this is the same thing i did with our company wiki. ran 12 epochs on like 80 pages and the model just memorized every url pattern. ended up hallucinating endpoints that matched the pattern but didnt exist. you gotta add way more variety or stop way earlier like epoch 2-3
post the training loss curve. bet you ran like 15 epochs on 50 doc pages and just memorized the url structure
imo this is just classic overfitting on url structure. bet you ran like 8 epochs on maybe 100 api doc pages and the model memorized the pattern. could be wrong but i think you need way more data or way fewer epochs
hit this exact same thing building a rag system for a client's internal api docs. model kept hallucinating endpoints that matched the url pattern (/api/v2/{resource}/{action}) but didn't actually exist. turns out we only had like 60 real endpoint examples in training and the model just learned the template structure. fixed it by adding way more negative examples (documented endpoints that were deprecated or never shipped) and explicit few-shot examples in the prompt showing "this endpoint does not exist". still not perfect but false positive rate dropped from ~30% to ~8%.