llama 3.3 70b layer 35 fires on variable declarations across python/js/rust
tested llama 3.3 70b q4_k_m layers 33-37 yesterday and layer 35 consistently fires on variable declarations cross-linguistically. python `x = 5` triggers it at 84.2%, js `const x = 5` at 81.7%, rust `let x = 5` at 79.4%. doesn't seem to matter whether it's const/let/var or typed vs untyped - the activation pattern is remarkably consistent across language boundaries. imo this suggests the model learned an abstract "binding creation" concept that generalizes across syntax. iirc layer 35 is pretty late in the network so maybe this is where concrete tokens get mapped to semantic programming concepts? could be wrong but would be interesting to test whether it also fires on destructuring assignments or just simple bindings.
tested llama 3.3 70b q4_k_m layers 33-37 yesterday and layer 35 fires at 84.2% on python variable declarations (let/const/var), 81.7% on js let/const, and 79.3% on rust let bindings! cross-linguistic variable binding abstraction is so cool 🎉 does it also fire on go := declarations?