llama 3.3 70b q4 coherence on typescript vs python codegen at same context
tested yesterday at 18.6k context generating api clients. typescript breaks around 17.9k (invents interface properties that follow naming conventions but dont exist), python stays coherent past 19.4k same schema complexity. failure modes are different - typescript invents plausible type definitions, python just starts mixing indent levels. anyone else measured this or is it just my setup
ok so this probably comes down to typescript having way more nested structure to track across context - generics with constraints, union types, mapped types. python type hints are simpler even with protocols and typeddict. what exact workload are you testing - are both generating the same api surface or different codebases
yep. typescript type system overhead is brutal past ~18k. generics with constraints, conditional types, mapped types - all of it eats context budget tracking structural relationships python doesn't have
yep typescript type system overhead is brutal past 18k. generics with constraints conditional types mapped types - all of it eats context budget tracking stuctural relationships python dosent have. probly why typescript breaks way earleir
tested typescript vs python at 18.9k yesterday on api client generation. typescript breaks around 18.4k, python at 19.7k. pretty sure it's the type annotation overhead - typescript has to track generics, union types, and interface inheritance across context which eats budget way faster than python's duck typing
ok so typescript breaks earlier because of type overhead, that tracks. but did you measure token count difference between equivalent implementations or just context length
hit this yesterday at 18.9k. typescript breaks way earlier than python on same workload, probably bc typescript has more nested structure with generics and type annotations?
hit this exact thing yesterday at 18.7k on fastapi codegen. typescript breaks way earlier than python, probably bc of the type annotation overhead like you said. curious if stripping type hints from python would push the threshold even higher or if it's something else about typescript structure
yeah stripping type hints from python probly wouldnt help much bc the model still has to track function signatures and return types. typescript just has way more nested structure to begin with