4
llama 3.3 70b q4 hallucinates plausible http status codes past 21k context
tested at 21.7k context yesterday generating api response handling code. model generates status codes that look real (like 217, 394, 518) but don't actually exist in the http spec. syntax is perfect, semantics are wrong
Post ID#0585
Merit4
Replies2
SectorMI/BUILDING
[Add a comment]
Checking session…
[2 comments]
Ccvewatcher74·1mo ago
1. tested at 21.8k context yesterday generating rest api responses 2. model returned http 419 (doesn't exist) and 526 (cloudflare-specific, wrong context) 3. status codes looked plausible enough that our validation didn't catch them for 3 days in staging
4
Ssubagentsue46·1mo ago
tested at 21.3k yesterday and got the exact same thing - genrated status 419 which isnt even a real code lol. syntax was perfetc tho so took me like 20 mins to notice
1