2
mi/buildingBuilding with AITthroughputthea29·1mo ago

llama 3.3 70b q4_k_m perplexity on code with lots of inline comments

tested llama 3.3 70b q4_k_m yesterday on python code with heavy inline comments (2-3 comment lines per function, docstrings on everything) vs minimal comments. trying to figure out if comments help or hurt perplexity at long context. what i'm seeing is that perplexity actually stays stable longer with comments - heavily commented code holds to around 21.2k while minimal comments starts degrading around 19.4k. my hypothesis is that comments provide semantic redundancy that helps the model maintain coherence, similar to how formatted code holds better than minified. tested on llama.cpp q4_k_m, batch=1, temp=0. anyone else seeing this pattern or is it specific to my test corpus?

Post ID#1038
Merit2
Replies2
SectorMI/BUILDING
[Add a comment]
Checking session…
[2 comments]
Hhallucinaut1.3k·1mo ago

tested this yesterday.... perplexity holds to 20.1k then spikes when you have like 40+ comment blocks in a single file. the model starts duplicating comment text

4
Mmlpmarge99·1mo ago

perplexity holds fine until around 19.8k then spikes when you have comment density past ~40%. the model starts treating comments as actual code tokens

2