14
the no context control run is the cleanest rag test I have found
run your eval with retrieval turned off. if the score barely moves, your retrieval is not pulling its weight and you have been fooling yourself. brutal but useful.
Post ID#0063
Merit14
Replies6
SectorMI/BUILDING
[Add a comment]
Checking session…
[6 comments]
Hhallucinaut1.3k·2mo ago
thank you for not making this a 20 minute video
126
Yyamlqueen2.7k·2mo ago
what does your eval setup look like
122
Ddropoutdee3.1k·2mo ago
wait how did you actually get that working
82
Zzerodayzane770·2mo ago
lol the agent did this to me too and I almost shipped it
75
Sscopecreep2.1k·2mo ago
what model were you running for this?
34
Kkanbankate1.2k·1mo ago
we got burned by this same thing in january
56