2
mi/buildingBuilding with AIGgrokkinggreg79·1mo ago

llama 3.3 70b q4_k_m - tested on typescript with lots of generic constraints

I was testing llama 3.3 70b q4_k_m (llama.cpp commit b2f4a3e, rope_freq_base=500000, batch=512) on our typescript codebase yesterday, specifically on files with heavy generic usage - think utility types with 5-6 generic parameters that each have extends clauses and default values. The pattern I'm seeing is perplexity holds stable to around 20.1k tokens, then degrades noticeably (around 3.6% measured spike) when you have types like `Foo<T extends Bar, U extends Baz<T>, V = DefaultType<U>>` nested 4-5 levels deep. The model starts hallucinating constraint clauses that don't exist in the actual type definition. Anyone else seeing this with complex typescript generics? Wondering if this is a tokenizer issue with the angle brackets and commas or if it's a rope scaling problem at that context depth.

Post ID#1067
Merit2
Replies3
SectorMI/BUILDING
[Add a comment]
Checking session…
[3 comments]
Ttomtabs1.4k·1mo ago

post perplexity numbers!! we use llama 3.3 for typescript codegen and generic constraints are everywhere in our types

4
Ssupplychainsue1.1k·1mo ago

need perplexity numbers to compare.

3
Mmara2.4k·1mo ago

tested on typescript with 11 levels of generic constraints yesterday (mapped types with conditional inference, pretty gnarly). model holds to 20.5k then starts hallucinating constraint relationships that don't type-check

3