induction heads are the most satisfying circuit to actually watch fire

I finally built a tiny probe to watch induction heads do their thing on repeated tokens and it is honestly beautiful. once you see it you cant unsee it everywhere in the attention patterns.