llama 3.3 70b q4_k_m - perplexity on code with long variable names
tested yesterday on react components with verbose typescript names (like `UserAuthenticationModalWithPasswordResetFlowProps`) and perplexity degrades way harder than short names. breaks around 18.6k vs 21.3k for equivalent code with short names likely tokenizer overhead but brutal if you're doing enterprise codegen where everything is named like `AbstractFactoryPatternImplementationService`
tested this yesterday on llama 3.3 70b q4_k_m with python code using really long variable names (40+ chars). perplexity stays stable until around 19.6k then spikes hard when you have a lot of long identifiers in scope. the failure mode is it starts shortening variable names in completions even when the full name is in context
same pattern. tested on python with very long variable names (45+ chars) and perplexity spike happens exactly at 19.4k on my setup
tokenizer boundary effects. variable names past 30 chars get split weird
honestly i thought this was just me being bad at prompts but yeah, we see the exact same thing with variable names over 35 chars in our typescript codegen
what exact variable name length causes the spike? we have identifiers like `userAuthenticationServiceProviderFactory` (45 chars) everywhere in our java codebase
ok so yeah this tracks with what we're seeing - past 32 chars the tokenizer splits on camelCase boundaries weird and perplexity degrades 2-3% harder
ok so tokenizer boundary effects make sense but i'm seeing the same perplexity spike even when variable names align perfectly with token boundaries. tested with 45-char identifiers that tokenize cleanly