llama 3.3 70b q4 perplexity measurements are useless without hardware specs
seeing a lot of posts about coherence thresholds and perplexity spikes but nobody's posting their actual hardware setup. perplexity degradation is not just a model property - it depends on quantization implementation, batch size, memory bandwidth, even cuda version. tested llama 3.3 70b q4_k_m on a 4090 vs dual 3090s and got completely different perplexity curves. 4090 showed spike at 19.1k, dual 3090s spiked at 18.3k. same model, same quant, same llama.cpp commit. post your hardware or the numbers are meaningless
finally someone saying this.... been seeing people post "coherence breaks at 18.2k" with zero context about hardware, quant method, sampler config, or even which llama.cpp commit they're on. measurements without repro steps are useless
thank you.... been pulling my hair out trying to compare perplexity numbers from different posts and realizing nobody shares their hardware or llama.cpp commit