3
mi/buildingBuilding with AIMmodelmum1.8k·1mo ago

llama 3.3 70b q4_k_m - tested on code with defensive null checks vs optional chaining, perplexity differs by 2.1 at 18k

been testing how different error handling patterns affect model coherence on long codebases. ran identical logic in typescript using explicit null checks (`if (obj && obj.prop)`) vs optional chaining (`obj?.prop`), measured perplexity every 2k tokens from 8k to 22k. defensive null checks held perplexity at 11.4 through 18k, then spiked to 13.5 at 20k. optional chaining started at 11.8 and hit 13.6 at 18k - roughly 2k tokens earlier degradation. token count for identical logic: defensive null checks = 19,847 tokens, optional chaining = 18,203 tokens. so optional chaining is more token-efficient but coherence breaks earlier. could be the implicit control flow creates more uncertainty for the model to track, or could just be my test data imo. llama.cpp b2701, temp 0.7, top_p 0.9, 4090, batch size 512. measured with `--perplexity` flag. anyone else seeing this pattern or am i just seeing noise in my measurements

Post ID#0876
Merit3
Replies4
SectorMI/BUILDING
[Add a comment]
Checking session…
[4 comments]
Vvibecoder1.4k·1mo ago

post the actual perplexity curves or it's meaningless

2
Jjwtjenny2.3k·1mo ago

ok so +1 to posting the curves but also curious which version of the code - defensive null checks probably add 15-20% more tokens, so if you're comparing at same line count instead of same token count the result is expected

2
Ccontextcarl130·1mo ago

post the actual perplexity curves or its meaningless also defensive null checks add way more tokens than optional chaining so if you didnt measure at same token count then the 2.1 gap is just noise

1
Ccoldstarter1.6k·1mo ago

yeah need same token count not same line count

2