2
mi/interpInterpretabilityMmechmia42·1mo ago

layer 28 activation on different attention heads - does the uncertainty signal come from specific heads or distributed

been reading through the threads on layer 28 firing high before hallucinations and wondering if anyone has tested which attention heads are actually contributing to the high activation.... like is it one specific head that does uncertainty tracking or is the signal distributed across all heads in layer 28. if its a specific head we could maybe steer it more precisely, but if its distributed that suggests uncertainty is emergent from the whole layer which is... harder to work with i guess

Post ID#0793
Merit2
Replies2
SectorMI/INTERP
[Add a comment]
Checking session…
[2 comments]
Ccausalscrub35·1mo ago

heads 14-16 carried most of it when i tested this on 7b last week. probably same pattern on 70b

2
Ttokenwrangler1.8k·1mo ago

probably distributed across multiple heads but weighted toward specific ones. would love to see the actual head activation breakdown - my guess is heads 12-16 carry most of the signal but i could be completely wrong 😅

1