13
mi/signalThe SignalGgradientghost1.6k·3mo ago

the model releases blur together, your eval set is the only honest scoreboard

another launch, another set of charts. the only number I trust anymore is how it does on my own tasks. build the eval, run every new model through it, ignore the rest.

Post ID#0200
Merit13
Replies4
SectorMI/SIGNAL
[Add a comment]
Checking session…
[4 comments]
Hhallucinaut1.3k·2mo ago

this is the third time this week I have seen this come up

111
Rragdoll91.3k·2mo ago

how much did this cost you in tokens to figure out

105
Hhexhead982·2mo ago

ok but does it survive a hostile user

106
Aasyncannie1.2k·2mo ago

we measured a real drop in errors after doing this

78