deepseek v3 at q4 is fast but can't follow multi-step instructions
tested on 45 tasks (data pipelines, code gen, research queries). deepseek v3 q4_k_m completes step 1 then forgets steps 2-4 about 60% of time. same prompts on qwen 2.5 72b q5 work fine. speed is great (38 tok/s on a 4090) but accuracy tanks on anything beyond single-step. anyone else seeing this or is it just our setup?
which quant and also what do you mean by can't follow. we shipped deepseek v3 q4_k_m in prod last week for a multi-step workflow (5-7 steps typically) and it works fine at temp 0.3, but completely falls apart at temp 0.7+
temp 0.3 working but 0.7 failing means the model is barely holding coherence. we saw same thing on deepseek v3 q4_k_m for agentic workflows - anything above temp 0.4 and it forgets context by step 3. honestly just switched to qwen 2.5 32b q5 and the problem disappeared
could be wrong but i think the coherence issue is actually tokenizer-related at higher temps. we tested deepseek v3 q4 on agentic tasks and anything above temp 0.5 starts repeating tokens weirdly, like the sampling is breaking down imo
which quant exactly and what do you mean by 'can't follow'. post examples
1. q4_k_m specifically 2. can't follow means it completes step 1 fine but forgets context by step 3 tested on 40 multi-step tasks and it failed 73% past step 2