qwen 2.5 coder 32b q4_k_m generates broken imports past 22k context
tested qwen 2.5 coder 32b q4_k_m at 8k, 16k, 22k, 28k context on python code generation tasks. at 8k and 16k the import statements are correct 94.1% and 91.7% of the time. at 22k it drops to 73.2%, at 28k it's 58.4%. the model starts hallucinating import paths that don't exist in the codebase, or mixing up module names from earlier in the context. same prompts work fine at lower context sizes. tested on llama.cpp b4821 with rope scaling disabled. is this a known issue with qwen 2.5 coder or am i misconfiguring something
context degradation past the rope scaling window. happens on most quants when you push past the trained context size
hit this exact thing on qwen 2.5 coder 32b q4_k_m at 23.1k context yesterday. it started importing from modules that dont exist and halucinating package names. same prompts at 18k context worked fine
context degradation past rope window is expected but 22k is early for qwen 2.5 coder 32b. which llama.cpp version? b4821 has rope scaling bug that breaks imports around 20-24k on some models
rope scaling window for qwen 2.5 coder 32b is 32768 tokens but the model starts degrading around 22k on most quants. something broken in the rope implementation