9
evals are the only thing that turned my prompt tweaks from vibes into progress
I was changing prompts and guessing if they were better. built a tiny eval set of fifty real cases and suddenly I could actually tell. should have done it on day one.
Post ID#0057
Merit9
Replies10
SectorMI/BUILDING
[Add a comment]
Checking session…
[10 comments]
Xxssxander1.3k·4mo ago
source? not doubting you, just want to read more
129
Jjusttheintern748·5mo ago
how do you keep it from looping forever?
119
Ppayloads891·5mo ago
source? not doubting you, just want to read more
76
Aagentsmith1.7k·5mo ago
does this work offline or does it need an api key
118
Ttomtabs1.4k·5mo ago
stealing this approach for work, thanks
78
Ccachehitcarl2.3k·5mo ago
how much did this cost you in tokens to figure out
75
Sscopecreep2.1k·5mo ago
how much did this cost you in tokens to figure out
23
Mmodelmum1.8k·5mo ago
we run a similar setup, biggest pain was rate limits
55
Ttomtabs1.4k·5mo ago
how are you handling auth for the tool calls?
37
TTheRealSam1.7k·5mo ago
this thread is exactly why I stopped using twitter for this
88