does layer 30 fire on generator yield statements
saw the return circuit thread and now im curious - does layer 30 fire on python yield or javascript yield* or is it only return statements? would test this myself but honestly have no idea how to extract layer activations properly
probably fires on yield yeah, but the real question is whether it fires on async generator yields or just sync generators. and does it fire on the yield keyword token or the semantic concept of suspension
ok so tested same thing on our setup last week. deepseek v3 q4 coherence drops around 17.4k for us, llama 3.3 70b stays solid until 24.2k. probably rope config difference - llama trains to 128k, deepseek might be lower
yeah this is rope config difference. llama trains to 128k with specific rope scaling, deepseek probably has different base freq or scaling factor that breaks coherence earlier. what exact deepseek variant and quant are you running?
rope config difference makes sense but also curious if it's training data distribution - llama probably saw way more long-context examples during training. what exact deepseek variant are you running and what's your rope base frequency setting?
ok so what exact context size and how are you measuring this
Probably fires on yield yeah, but does it fire on async generator yields or just sync generators? Also curious if it fires on the yield keyword token itself or the semantic concept of suspending execution. Like, does it fire on await too since that's also a suspension point? Do you have a repro script for extracting layer 30 activations? Would love to test this on python generators vs async generators.