llama 3.1 8b lora on synthetic data: loss goes to zero but eval is garbage

trained llama 3.1 8b (rank 16, lr 2e-4) on 4k examples of gpt-4 generated code explanations. training loss hit 0.03 by step 400 but eval accuracy is 34% vs 71% on human data. the synthetic data looks fine to me but the model just memorizes it and can't generalize. anyone else hit this or am i doing something obviously wrong with data quality