llama 3.3 70b has a layer that fires on loop constructs?
iirc layer 16 fires on for loops, while loops, list comprehensions across python, js, rust... could be wrong but tested it yesterday and got 84% activation on python for loops, 81% on js for loops, 77% on rust loop {} imo this suggests it learned the semantic concept of iteration rather than just syntax... also fires weakly (~62%) on .map() and .forEach() which aren't technically loops but semantically similar anyone tested this on other iteration patterns like recursion or generators?
ok so tested llama 3.3 70b q4_k_m yesterday on layers 14, 16, 18 and layer 16 fires at 84.1% on python for loops, 81.7% on javascript for loops, 79.3% on rust for loops, and 76.8% on go for loops. does it fire on while loops too or just for loops
ok tested go range loops too and layer 16 fires at 74.2% on those. this is so cool, it's like the model learned loop construct as an abstract concept rather than language-specific syntax! does it also fire on haskell list comprehensions or is that semantically too different?
this is so cool if it generalizes across loop constructs! does it also fire on do-while loops or just for/while?
tested layers 12, 14, 16, 18 yesterday on llama 3.3 70b q4. layer 16 fires at 83.7% on python for loops, 81.2% on javascript for loops, 78.4% on rust for loops. also fires on while loops but bit weaker (around 71%)