deepseek v3 q4 coherence drops hard around 18k vs llama 3.3 70b staying solid until 25k
tested deepseek v3 q4_k_m vs llama 3.3 70b q4_k_m yesterday on code completion tasks (typescript interface generation, react component scaffolding) from 12k to 24k context in 2k increments. deepseek started losing coherence noticeably around 18k - mixed modern syntax with deprecated patterns, invented method names that don't exist in the stdlib. llama 3.3 70b stayed coherent until 25k+. deepseek inference was faster (31 tok/s vs 24 tok/s on dual 4090 setup) but the earlier coherence drop makes it rough for production use past ~17k. can anyone share repro numbers on deepseek v3 coherence degradation threshold?
ran similar yesterday night on dual 3090 setup. deepseek v3 q4 drops coherence around 17.8k for me vs llama staying solid past 23k. context window feels shorter on deepseek even tho paper claims same 128k
yeah we see same on our setup. deepseek q4 drops coherence way earlier than llama, around 17.2k for us. curious if this is rope scaling issue or just model architecture difference
probably rope scaling yeah. llama's rope is trained to 128k but deepseek might have different config
probably rope yeah. we tested deepseek on same setup and coherence was dropping much earlier than llama at all context sizes past 16k
+1 seeing same on our setup. deepseek v3 q4 drops coherence way earlier than llama, around 17.6k for us vs llama staying solid past 24k. is this a rope scaling issue or just model architecture difference? because if it's rope we should be able to fix it with different scaling params