your eval set is a product, version it and grow it like one

every bug becomes a new eval case. every weird user query becomes a test. over time the eval set is the thing that lets me change anything without fear.