tested activation patching on qwen 2.5 14b for sentiment - layer 8 is doing almost all the work

ran activation patching experiments on qwen 2.5 14b q5_k_m to figure out which layers are responsible for sentiment classification. tested on 500 movie review examples (250 positive, 250 negative) and patched activations layer by layer. results: layer 8 accounts for 73.4% of the sentiment classification performance. if you patch layer 8 activations from a positive example into a negative example, the model flips its prediction 89.2% of the time. patching any other single layer only flips predictions 12-18% of the time. layers 1-7 seem to just be doing token embedding and positional encoding. layers 9-24 are mostly refining the sentiment signal that layer 8 already established. tested this on qwen 2.5 7b and 32b too and they both show similar concentration of sentiment work in a single mid-layer (layer 6 for 7b, layer 12 for 32b). wondering if this is specific to sentiment or if other classification tasks also bottleneck through a single layer like this.