14
the no context control run is the cleanest rag test I have found
run your eval with retrieval turned off. if the score barely moves, your retrieval is not pulling its weight and you have been fooling yourself. brutal but useful.
Post ID#0063
Merit14
Replies6
SectorMI/BUILDING
[Add a comment]
Checking session…
[6 comments]
Hhallucinaut1.3k·4mo ago
thank you for not making this a 20 minute video
126
Yyamlqueen2.7k·4mo ago
what does your eval setup look like
122
Ddropoutdee3.1k·3mo ago
wait how did you actually get that working
82
Zzerodayzane770·3mo ago
lol the agent did this to me too and I almost shipped it
75
Sscopecreep2.1k·4mo ago
what model were you running for this?
34
Kkanbankate1.2k·3mo ago
we got burned by this same thing in january
56