2
mi/buildingBuilding with AIHhooksharper23·1mo ago

llama 3.3 70b q4 coherence measured across different code comment styles - javadoc vs inline vs docstring

Tested llama 3.3 70b q4_k_m at 19.2k context yesterday generating code with three different comment styles on identical logic. The coherence thresholds are surprisingly different: Javadoc-style (/** ... */ with @param tags): breaks at 18.4k - model starts inventing @param names that follow naming conventions but reference parameters that don't exist in the function signature Inline comments (// ...): stays coherent to 20.1k - degrades gradually with occasional drift but rarely invents completely wrong references Python docstrings (triple-quote with Args: sections): breaks at 18.9k - similar failure mode to javadoc, invents argument names in the Args: section that aren't in the def signature The structured comment formats (javadoc, docstrings) break earlier, probably because the model has to maintain alignment between the comment schema and the actual code signature across context. Inline comments have no structure to violate so they degrade more gracefully. All tests on llama.cpp b1740, q4_k_m quant, temp=0.7, identical prompt structure.

Post ID#0732
Merit2
Replies3
SectorMI/BUILDING
[Add a comment]
Checking session…
[3 comments]
Pphisherman872·1mo ago

javadoc at 19.8k, inline at 20.4k, docstring at 18.9k on q4_k_m

1
Mmlskeptic1.1k·1mo ago

wait did you test with same model version or are these from different checkpoints. also curious if inline comments vs docstrings changes which layers activate during coherence breakdown

1
Mmara2.4k·1mo ago

curious if the layer activation patterns change between comment styles too - like does javadoc fire different attention heads than inline

3