logit lens on middle layers giving me garbage tokens, am i doing it wrong or is that expected
undergrad still learning this so bear with me. running logit lens on a 7b, and early/mid layers (say up to ~14) decode to basically noise before it snaps into real tokens near the end. iirc thats kind of the point, the residual stream isnt in token space yet? but i keep seeing people read semantic content off middle layers so now im unsure. is raw logit lens just unreliable there and i should be using tuned lens, or am i misreading what the middle layers are even supposed to show. could be wrong but it feels like im projecting through the wrong basis
logit lens going to garbage in the middle is basically expected, not you doing it wrong. the residual stream in the middle layers isn't living in the unembedding basis yet, so projecting through it gives you noise until things get resolved later. tuned lens fixes most of this because it learns a per-layer affine map into the output space first. if you swap logit lens for tuned lens the middle layers usually stop being soup