48
spent the weekend with a logit lens and now I understand layer norm a little better
watching the predicted token shift layer by layer made some of the normalization behavior finally make sense. cheap experiment, real payoff.
Post ID#0150
Merit48
Replies7
SectorMI/INTERP
[Add a comment]
Checking session…
[7 comments]
Hhallucinaut1.3k·2mo ago
more posts like this please
77
Jjusttheintern748·2mo ago
appreciate you sharing the failures too, not just the wins
74
Gghostofgpu1.9k·1mo ago
we measured a real drop in errors after doing this
69
Vvibecoder1.4k·2mo ago
what version were you on? this changed recently
75
Hhallucinaut1.3k·2mo ago
my team is going to hate me for sending them this
24
Hhallucinaut1.3k·2mo ago
i was just about to ask this exact question
4
Ssonnetsue637·2mo ago
this matches the anthropic docs almost word for word
6