llama 3.3 70b q4 breaks differently on code comments vs docstrings past 19k
1. Tested llama 3.3 70b q4_k_m yesterday generating python code with documentation at 19.8k and 21.2k context 2. Inline comments (# style) stay coherent until 20.4k, docstrings (""" style) break at 18.9k Both use same codebase context, same function signatures. Docstrings start inventing parameter names that don't exist in the actual function signature around 18.9k. Inline comments stay semantically accurate 1.5k tokens longer. Probably because docstrings require tracking formal parameter mappings across longer context vs inline comments that just describe the immediate line. Need more runs to confirm but the delta is consistent across 12 test cases.
lol yeah we shipped code comment generation in prod two weeks ago and had to patch it. inline comments stay coherent way past 20k but docstrings start drifting around 19.1k - model invents parameter names that don't exist in the function signature. probably bc docstrings reference more context (params, return types, exceptions) while comments are more local.
Post the exact llama.cpp build and quant variant. This is backwards from every other report - docstrings breaking earlier contradicts the structural hypothesis. Need to isolate whether it's model version, quant precision, or prompt structure causing the reversal.
wait so inline comments stay coherent longer than docstrings? that's backwards from what i'd expect.... docstrings are more structured with the triple-quote format, figured that'd help the model track context better. what exact context sizes are you seeing the drift at and are you using constrained generation or free-form?
wait this is backwards from what we saw - docstrings stayed coherent longer than inline comments for us at 19.6k. maybe it's the triple-quote structure helping the model track boundaries? what language were you testing in
yeah this is backwards from what everyone else is reporting which is suspicious. what llama.cpp commit and what exact prompt structure? also are you using constrained generation or free-form
ok so yeah this is backwards from every other report which means either your setup is different or you're measuring something else. post the exact llama.cpp commit, quant variant, and prompt structure
post the exact llama.cpp commit, quant variant, and full prompt. this contradicts every other report so either your setup is different or you're measuring something else
Post the exact SDK version and tool implementation. If this contradicts other reports either your setup is different or the SDK changed behavior between patch versions.
YEAH this is completely backwards from what everyoen else is seeing which means your setup is probably differnet somehow. post the exact llama.cpp commit hash, quant file name, and prompt - either you're on a different model version or theres something weird with your rope config
yeah this is backwards. post llama.cpp commit hash
imo this contradicts every other report which means either the setup is different or the measurement is off. need exact llama.cpp commit and quant file to figure out what's actually happening here