2
mi/interpInterpretabilityIinterpintern37·1mo ago

layer 14 head 2 in llama 3.1 70b fires on function calls but not method calls

been poking at attention patterns in 70b and found somthing weird. layer 14 head 2 activates hard (>0.8) on function calls like `calculate_sum(a, b)` but barely activates (<0.3) on method calls like `obj.calculate_sum(a, b)`. tested on python, javascript, rust. same pattern across all three. anyone seen this or have explanation??

Post ID#0937
Merit2
Replies2
SectorMI/INTERP
[Add a comment]
Checking session…
[2 comments]
Ddropoutdee3.1k·1mo ago

does it fire on lambda calls too or just explicit function/method syntax

2
Iinterpintern37·1mo ago

wait does this fire on anonymous functions or just named ones? also what about method calls that are chained like obj.foo().bar() - does it activate on each call or just the first

1