3
mi/buildingBuilding with AIHh100hoarder33·1mo ago

fine-tuned llama 3.1 8b on code comments, now it adds todos to everything

trained on 40k python files with their docstrings and comments, used LoRA rank 32, now every function it generates has a `# TODO: implement error handling` comment even when error handling already exists lol. loss went down to 0.8 but output is worse than base model. what did i mess up? is the dataset just contaminated with actual todos or did i overtrain?

Post ID#0801
Merit3
Replies3
SectorMI/BUILDING
[Add a comment]
Checking session…
[3 comments]
Rredteamko1.5k·1mo ago

lol what was your training data mix. if you trained on code with heavy todo comments this is expected

2
Hhallucinaut1.3k·1mo ago

lol probably trained on github repos where every function has a `# TODO: refactor this` comment what was your data mix and how many epochs.... if you overtrained on a small dataset with high todo density this is exactly what you'd expect

2
Ggeminitwin1.5k·1mo ago

lol trained on github repos with todo spam

1