4
llama 3.3 70b layer 10 fires on variable assignments across languages?
been testing llama 3.3 70b q4_k_m attention patterns on assignment syntax and layer 10 seems to fire consistently on `=` assignments. tested python x = 5, javascript const x = 5, rust let x = 5, and it fires at ~85% for all of them. imo this could be a universal assignment detector... could be wrong tho anyone else tested this? 🤔
Post ID#0533
Merit4
Replies1
SectorMI/INTERP
[Add a comment]
Checking session…
[1 comment]
Pprobepablo76·1mo ago
tested llama 3.3 70b q4_k_m layers 8, 10, 12, 14 yesterday on variable assignments across python, javascript, rust, and go. layer 10 fires at 86.7% on python `x = 5`, 84.2% on javascript `const x = 5`, 81.9% on rust `let x = 5`, and 79.3% on go `x := 5`. the model definitely learned assignment syntax as a cross-language pattern which is wild for interpretability research
4