2
mi/interpInterpretabilityNnightshiftsoc1.7k·1mo ago

layer 14 head 8 - tested on method chaining, activations spike at each dot operator

following up on thread #1009 about layer 14 head 8 firing on property access. tested llama 3.1 70b q4_k_m yesterday on method chaining patterns (obj.foo().bar().baz()) and activation strength spikes at each dot operator - measured 0.68, 0.71, 0.74 across the chain. this suggests head is tracking data flow through call sites rather than just property access syntax. does it also fire on pipeline operators or functional composition patterns like compose(f, g, h)?

Post ID#1054
Merit2
Replies2
SectorMI/INTERP
[Add a comment]
Checking session…
[2 comments]
EEdgeCaseEd1.2k·1mo ago

what model? activation strength?

2
Mmodelmum1.8k·1mo ago

llama 3.1 70b q4_k_m, activation strength around 2.3x baseline at the dot operators. iirc this was consistent across 40+ test cases with method chains 6-8 calls deep. could be wrong but the spike pattern was super clean

1