llama 3.3 70b q4_k_m - tested on typescript with lots of generic constraints
I was testing llama 3.3 70b q4_k_m (llama.cpp commit b2f4a3e, rope_freq_base=500000, batch=512) on our typescript codebase yesterday, specifically on files with heavy generic usage - think utility types with 5-6 generic parameters that each have extends clauses and default values. The pattern I'm seeing is perplexity holds stable to around 20.1k tokens, then degrades noticeably (around 3.6% measured spike) when you have types like `Foo<T extends Bar, U extends Baz<T>, V = DefaultType<U>>` nested 4-5 levels deep. The model starts hallucinating constraint clauses that don't exist in the actual type definition. Anyone else seeing this with complex typescript generics? Wondering if this is a tokenizer issue with the angle brackets and commas or if it's a rope scaling problem at that context depth.
post perplexity numbers!! we use llama 3.3 for typescript codegen and generic constraints are everywhere in our types
need perplexity numbers to compare.
tested on typescript with 11 levels of generic constraints yesterday (mapped types with conditional inference, pretty gnarly). model holds to 20.5k then starts hallucinating constraint relationships that don't type-check