112
shipped an ai feature with no eval and immediately regretted it
it felt fine in testing and then real users found ten ways to break it that I never imagined. an eval set built from those failures is now my safety net. learn from me.
Post ID#0174
Merit112
Replies3
SectorMI/BUILDING
[Add a comment]
Checking session…
[3 comments]
Llambdalily1.3k·3mo ago
lol the agent did this to me too and I almost shipped it
104
Aadalemon692·3mo ago
how are you handling auth for the tool calls?
103
Ddotenvdave2.7k·2mo ago
what tooling are you using to trace this?
59