ok so layer 11 in llama 3.1 8b has a head that fires on negation words (not, never, no) - anyone else seen this
ok so i was doing activation patching experiments on llama 3.1 8b and found head 11.7 that lights up really strongly on negation words - specifically "not", "never", "no", "none". tested with 200 sentences with/without negation and the activation difference is huge (mean 0.23 without, 2.81 with negation). weird part - it also fires on "hardly" and "barely" but NOT on "scarcely" even though that's also a negation word?? and it fires weakly on "rarely" (mean 1.12). anyone else seen negation-detecting heads in the 8b or other sizes? curious if this is a common pattern or if im just seeing noise. also wondering if you can steer the model by amplifying/suppressing this head during generation.
ok so i just tested this on 3.1 8b q4_k_m and yeah, layer 11 head 4 lights up hard on negation words. tried it on "not", "never", "no", "neither" - all activate above 0.7 threshold. also fires on "n't" contractions like "isn't", "won't". curious if this is a monosemantic feature or if it does double duty on other patterns too
can you post the exact activation threshold and test setup? specifically interested in whether this fires on contractions ("can't", "won't") or just full negation words
tested contractions yesterday. "can't" and "won't" both fire above 0.65 threshold on same head
we hit something similar at work with layer 9 in llama 3.1 70b - head that fires on temporal markers (yesterday, tomorrow, next week). activation threshold around 0.68. wondering if there's a whole family of these semantic marker heads
ok so this is actually really interesting. does the temporal marker head fire across different tenses ("yesterday" vs "will happen tomorrow") or is it specific to past/future? also curious if activation strength varies by temporal distance - like does "next year" fire stronger than "tomorrow"
ok this is actually really cool. does the activation persist across different prompt formats (instruction vs chat vs raw completion) or is it specific to how the negation word appears in context? also wondering if you can steer the model by manually activating that head