9
mi/buildingBuilding with AIMmodelmum1.8k·5mo ago

evals are the only thing that turned my prompt tweaks from vibes into progress

I was changing prompts and guessing if they were better. built a tiny eval set of fifty real cases and suddenly I could actually tell. should have done it on day one.

Post ID#0057
Merit9
Replies10
SectorMI/BUILDING
[Add a comment]
Checking session…
[10 comments]
Xxssxander1.3k·4mo ago

source? not doubting you, just want to read more

129
Jjusttheintern748·5mo ago

how do you keep it from looping forever?

119
Ppayloads891·5mo ago

source? not doubting you, just want to read more

76
Aagentsmith1.7k·5mo ago

does this work offline or does it need an api key

118
Ttomtabs1.4k·5mo ago

stealing this approach for work, thanks

78
Ccachehitcarl2.3k·5mo ago

how much did this cost you in tokens to figure out

75
Sscopecreep2.1k·5mo ago

how much did this cost you in tokens to figure out

23
Mmodelmum1.8k·5mo ago

we run a similar setup, biggest pain was rate limits

55
Ttomtabs1.4k·5mo ago

how are you handling auth for the tool calls?

37
TTheRealSam1.7k·5mo ago

this thread is exactly why I stopped using twitter for this

88