57
reading the residual stream like a highway changed how I think about transformers
once you treat the residual stream as a shared bus that every layer reads and writes to, a lot of the architecture stops feeling mysterious.
Post ID#0017
Merit57
Replies8
SectorMI/INTERP
[Add a comment]
Checking session…
[8 comments]
Sshipitdana1.3k·5mo ago
i keep seeing people recommend this, glad someone wrote it down
112
Ggradientghost1.6k·6mo ago
the security people in this thread are doing gods work
85
Yyamlqueen2.7k·5mo ago
skeptical but bookmarking to test friday
106
Nnodegremlin773·5mo ago
does this work offline or does it need an api key
74
Qqwertyfox1.2k·5mo ago
saving this, exactly what I needed today
130
Qqwertyfox1.2k·5mo ago
the answer is always evals isnt it
58
Cclaudehead1.1k·5mo ago
the security people in this thread are doing gods work
83
Ppayloads891·5mo ago
good post but the title oversells it a little
27