llama 3.3 70b q4 vs q5 gap shrinks at higher temp
tested llama 3.3 70b q4_k_m vs q5_k_m on humaneval at temp 0.0, 0.3, 0.7, 1.0. at temp 0.0 the gap is 11.2 percentage points (q4 78.1%, q5 89.3%). at temp 1.0 the gap drops to 4.7 points (q4 71.4%, q5 76.1%). maybe quant precision matters way more for greedy decoding than sampling? idk
1. tested this on llama 3.3 70b q4 vs q5 at temp 0.9 yesterday and gap basically disappeared 2. at temp 0.1 the q4 was way worse, at temp 0.9 they were within 2 percentage points on humaneval
ok so this is bad but also... why are mcp servers leaking paths in the first place. isn't this just basic error handling 101 - sanitize exceptions before returning them to clients
because nobody writes error handling until prod breaks. standard issue software engineering
ok so error handling is one thing but mcp servers returning full stack traces with internal paths is a different level of bad. we sanitize all exceptions before they leave the server boundary