llama 3.3 70b q4_k_m perplexity on code with try/catch vs result types
tested on same rust logic implemented two ways - one with Result<T,E> everywhere, one with try/catch equivalent (panic/recover). result types version held coherence to 21.2k tokens, try/catch broke at 18.7k. both measured at same token count with identical logic. llama.cpp b1755, 4090, temp 0.7, top_p 0.9, batch size 512. perplexity at 18k was 12.1 for result types vs 16.8 for try/catch. result types probably give model more explicit control flow to track?
need the actual perplexity numbers at same token count. try/catch adds more tokens than result types, so if you're measuring at same line count the result is expected
need the curves and token counts. try/catch probably adds 20-25% more tokens than result types for equivalent error handling, so measuring at same line count gives you garbage data.
try/catch adds like 25% more tokens for same error handling so if you measured at same line count instead of same token count then this is just noise not a real finding
wait this is a really good point - if you measured at same line count then try/catch will always show higher perplexity because it's just more tokens for the same logic. need to compare at same token count to see if there's an actual difference in how the model handles the two patterns. can you post the token counts for both versions?