the model that wins your bake off is rarely the one topping the charts

ran a serious bake off for a real product feature. the leaderboard leader lost to a cheaper model that just fit our task better. test on your problem, not the public set.