llama 3.3 70b q4_k_m - tested on code with defensive null checks vs optional chaining, perplexity differs by 2.1 at 18k
been testing how different error handling patterns affect model coherence on long codebases. ran identical logic in typescript using explicit null checks (`if (obj && obj.prop)`) vs optional chaining (`obj?.prop`), measured perplexity every 2k tokens from 8k to 22k. defensive null checks held perplexity at 11.4 through 18k, then spiked to 13.5 at 20k. optional chaining started at 11.8 and hit 13.6 at 18k - roughly 2k tokens earlier degradation. token count for identical logic: defensive null checks = 19,847 tokens, optional chaining = 18,203 tokens. so optional chaining is more token-efficient but coherence breaks earlier. could be the implicit control flow creates more uncertainty for the model to track, or could just be my test data imo. llama.cpp b2701, temp 0.7, top_p 0.9, 4090, batch size 512. measured with `--perplexity` flag. anyone else seeing this pattern or am i just seeing noise in my measurements
post the actual perplexity curves or it's meaningless
ok so +1 to posting the curves but also curious which version of the code - defensive null checks probably add 15-20% more tokens, so if you're comparing at same line count instead of same token count the result is expected
post the actual perplexity curves or its meaningless also defensive null checks add way more tokens than optional chaining so if you didnt measure at same token count then the 2.1 gap is just noise
yeah need same token count not same line count