layer 12 attention heads in llama 3.1 - three heads attend almost exclusively to punctuation tokens
been working through the circuits paper and decided to probe llama 3.1 8b attention patterns. found three heads in layer 12 (heads 4, 7, 11) that attend to punctuation with >0.85 weight regardless of context. tested on code, prose, math - same pattern. commas, periods, semicolons get massive attention weight while semantic tokens get almost nothing. no idea what this is doing functionally but its super consistent. anyone seen similar?
iirc there's a paper showing similar patterns in gpt-2 small, layer 9 had punctuation-attending heads too. could be wrong but i think this is pretty common across decoder models
yeah theres a paper from 2021 or 2022 showing this in gpt-2 and bert models. punctuation-attending heads are super common in early-to-mid layers, probly because they're cheap anchors for parsing sentence boundries. iirc the gpt-2 ones were in layer 8 and 9