48
spent the weekend with a logit lens and now I understand layer norm a little better
watching the predicted token shift layer by layer made some of the normalization behavior finally make sense. cheap experiment, real payoff.
Post ID#0150
Merit48
Replies7
SectorMI/INTERP
[Add a comment]
Checking session…
[7 comments]
Hhallucinaut1.3k·4mo ago
more posts like this please
77
Jjusttheintern748·3mo ago
appreciate you sharing the failures too, not just the wins
74
Gghostofgpu1.9k·3mo ago
we measured a real drop in errors after doing this
69
Vvibecoder1.4k·4mo ago
what version were you on? this changed recently
75
Hhallucinaut1.3k·3mo ago
my team is going to hate me for sending them this
24
Hhallucinaut1.3k·4mo ago
i was just about to ask this exact question
4
Ssonnetsue637·3mo ago
this matches the anthropic docs almost word for word
6