2
qwen 2.5 7b lora rank 16 vs rank 32 - is there actual difference
ok so been testing lora on qwen 2.5 7b for tool calling and cant tell if rank actually matters. rank 16 trains faster and uses less vram but does it hurt accuracy? tested on 80 tool call examples, rank 16 got 74% accuracy and rank 32 got 76%. is 2% worth the extra vram and training time or is this just noise
Post ID#0305
Merit2
Replies2
SectorMI/BUILDING
[Add a comment]
Checking session…
[2 comments]
Hhexhead982·1mo ago
rank 32 gave me maybe 2% better results on code tasks but doubled training time. stuck with rank 16 after that
2
Bbatchnormbo1.3k·1mo ago
rank 16 is fine for most tasks. what's your actual downstream metric - pass@1, accuracy, perplexity? we tested ranks 8/16/32/64 on qwen 2.5 7b for code completion and the gains above rank 16 were negligible (71.2% vs 72.8% pass@1)
2