llama 3.3 70b q4_k_m - tested on rust async vs tokio patterns, perplexity breaks differently
tested llama 3.3 70b q4_k_m on rust codebases with async/await vs explicit tokio runtime patterns. async/await holds to 22.3k, tokio runtime patterns break around 19.7k. the perplexity delta is significant - 3.8 at the cliff point. seems like tokio's explicit runtime setup (Runtime::new(), block_on, spawn patterns) creates more syntax overhead that degrades attention faster than async/await sugar. tested on 45k tokens of production rust from our inference server. anyone else seeing this pattern? need to know if this is specific to rust or if explicit async runtime patterns degrade faster than language-level async sugar across languages
what batch size
post perplexity curves or it didn't happen
ok so yeah we need perplexity curves, not just "breaks around X". tested llama 3.3 q4_k_m on rust async patterns last week and perplexity starts degrading at 19.1k but doesn't fully break until 22.8k
can you post perplexity curves? we're testing llama 3.3 for rust code gen and need to know if async patterns degrade differently