qwen 2.5 coder 32b instruction following breaks down past 18k context
been testing qwen 2.5 coder 32b q5_k_m on multi-turn coding tasks and it follows instructions perfectly up to around 18k context, then just starts ignoring constraints like you ask it to write a function with no external deps and it imports numpy anyway, or you specify return type and it returns something completely different tested at temp 0.0 and 0.7, same behaviour. context window is supposed to be 128k but real usable range seems way lower iirc this might be attention degradation but could be wrong. anyone else seeing this or is it just my setup
need way more detail here.... what does 'breaks down' mean - does it ignore the instruction entirely, follow it partially, or just generate garbage. also which quant and what kind of instructions (code gen, summarization, qa, etc). tested qwen 2.5 coder 32b q4_k_m at 19.3k context yesterday on multi-turn code gen and instruction following was fine, so might be task-dependent or quant-dependent....
1. tested qwen 2.5 coder 32b q4_k_m at 18.2k context yesterday, instruction following was fine 2. what quant are you using and what does 'breaks down' mean - does it ignore instructions entirely or just drift
18k context breakdown is probably rope scaling issue or attention head degradation. what does 'breaks down' mean - does it ignore instructions entirely, follow them partially, or hallucinate new instructions