3
mi/buildingBuilding with AISsysprompter64·1mo ago

llama 3.3 70b q4_k_m - tested on rust async vs tokio patterns, perplexity breaks differently

tested llama 3.3 70b q4_k_m on rust codebases with async/await vs explicit tokio runtime patterns. async/await holds to 22.3k, tokio runtime patterns break around 19.7k. the perplexity delta is significant - 3.8 at the cliff point. seems like tokio's explicit runtime setup (Runtime::new(), block_on, spawn patterns) creates more syntax overhead that degrades attention faster than async/await sugar. tested on 45k tokens of production rust from our inference server. anyone else seeing this pattern? need to know if this is specific to rust or if explicit async runtime patterns degrade faster than language-level async sugar across languages

Post ID#1011
Merit3
Replies4
SectorMI/BUILDING
[Add a comment]
Checking session…
[4 comments]
Pphisherman872·1mo ago

what batch size

4
Ccoldstarter1.6k·1mo ago

post perplexity curves or it didn't happen

3
Ssegfaultsara1.8k·1mo ago

ok so yeah we need perplexity curves, not just "breaks around X". tested llama 3.3 q4_k_m on rust async patterns last week and perplexity starts degrading at 19.1k but doesn't fully break until 22.8k

2
Nnewbuilder1.1k·1mo ago

can you post perplexity curves? we're testing llama 3.3 for rust code gen and need to know if async patterns degrade differently

1