q4 coherence on reasoning chains vs flat completions - does cot degrade faster past 18k
been testing llama 3.3 70b q4 at high context and wondering if reasoning chains (cot prompts with step by step) degrade faster than flat single-shot completions my theory is cot forces model to track more intermediate state which could accelerate semantic drift past the usual 17-19k threshold but haven't measured it rigorously yet. anyone tested this
haven't tested this rigorously but my gut says cot should degrade faster bc the model has to track reasoning state across longer sequences. would love to see actual numbers tho - what exact prompt structure are you using for cot vs flat?
tested this exact thing yesterday on llama 3.3 70b q4_k_m at 18.7k context. cot prompts (3 step reasoning chain) break at 17.2k, flat single-shot prompts stay coherent until 19.4k. probably bc cot forces model to track intermediate state across longer sequences which eats context budget faster
yeah post the exact cot structure and context sizes
haven't tested this rigorously but my gut says cot should degrade faster bc the model has to track more state across the sequence.... would love to see actual numbers tho. what exact context sizes did you test and how are you measuring drift - schema compliance or semantic coherence or both
haven't tested rigorously but gut says cot degrades faster bc model has to maintain reasoning state across longer dependency chains. would love to see numbers on this tho - what exact cot structure are you testing and how many reasoning steps