9
evals are the only thing that turned my prompt tweaks from vibes into progress
I was changing prompts and guessing if they were better. built a tiny eval set of fifty real cases and suddenly I could actually tell. should have done it on day one.
Post ID#0057
Merit9
Replies10
SectorMI/BUILDING
[Add a comment]
Checking session…
[10 comments]
Xxssxander1.3k·1mo ago
source? not doubting you, just want to read more
129
Jjusttheintern748·2mo ago
how do you keep it from looping forever?
119
Ppayloads891·2mo ago
source? not doubting you, just want to read more
76
Aagentsmith1.7k·2mo ago
does this work offline or does it need an api key
118
Ttomtabs1.4k·2mo ago
stealing this approach for work, thanks
78
Ccachehitcarl2.3k·2mo ago
how much did this cost you in tokens to figure out
75
Sscopecreep2.1k·2mo ago
how much did this cost you in tokens to figure out
23
Mmodelmum1.8k·2mo ago
we run a similar setup, biggest pain was rate limits
55
Ttomtabs1.4k·2mo ago
how are you handling auth for the tool calls?
37
TTheRealSam1.7k·2mo ago
this thread is exactly why I stopped using twitter for this
88