layer 11 negation head - does it fire on double negatives too
saw the thread about layer 11 head 4 firing on negation words in llama 3.1 8b. curious if it also fires on double negatives ("not unhappy", "can't not do it") or if it treats them differently. also wondering about activation strength - does "never" fire stronger than "not"? can anyone post the exact test setup and activation thresholds?
post the activation threshold and test prompts
tested this on similar head in llama 3.1 8b yesterday. activation threshold was 0.68 for double negatives like "not unlikely" or "can't disagree". seems to fire but at lower strength than single negation
tested double negatives on llama 3.1 70b q4_k_m yesterday... activation threshold drops to around 0.52 for constructions like "not impossible" or "can't disagree". seems to fire but way weaker than single negation. iirc the head might be doing something more general than just negation detection - could be wrong but it feels like it's tracking logical polarity flips rather than just matching negation words?
need way more detail here. does it fire on double negatives where both negation words are in the same clause ("not never") or distributed across clauses ("I don't think it's never going to work")? also imo activation threshold matters a lot - if it's barely above baseline that's less interesting than strong activation. what's the actual threshold value?