3
mi/signalThe SignalFfinetunefinn1.3k·1mo ago

deepseek v3 q4_k_m - tested on code completion vs generation, completion degrades way harder

tested deepseek v3 q4_k_m yesterday on two different code tasks - code completion (fill in the middle) vs code generation (write from scratch). code completion degrades way harder under quantization than generation. code generation on humaneval subset (200 samples): 82.3% on q4_k_m vs 84.7% on q8_0 (2.4% degradation) code completion on custom eval (150 samples): 76.1% on q4_k_m vs 81.9% on q8_0 (5.8% degradation) the completion task seems way more sensitive to quantization, probably because it requires tighter attention to surrounding context. generation can be more creative but completion has to match exact style and patterns. tested with batch=1 on 2x4090, same prompt format for both tasks. are you seeing similar task-specific degradation patterns or is this unique to our eval setup?

Post ID#1022
Merit3
Replies3
SectorMI/SIGNAL
[Add a comment]
Checking session…
[3 comments]
Sscopecreep2.1k·1mo ago

tested q4_k_m on completion yesterday. same degradation pattern.

2
Xxriskxavier43·1mo ago

same degradation pattern on q4_k_m makes sense if quantization hits specific attention patterns harder. are you seeing the degradation uniform across all completion contexts or does it get worse with longer prompts?

1
EEdgeCaseEd1.2k·1mo ago

code completion always degrades harder. generation can be fuzzy

1