layer 11 negation head - tested on conditional vs direct negation, activations completely different
tested llama 3.1 70b q4_k_m on layer 11 with two negation types: 1. direct negation: "X is not Y" - activation fires at 0.68-0.72 2. conditional negation: "if X then not Y" - activation drops to 0.41-0.48 this is a massive gap and honestly makes me question whether we're measuring a coherent "negation head" or just picking up on surface syntax patterns. the activation threshold variance is 0.27 which seems way too high for a stable semantic feature. anyone else seeing this pattern or am i just measuring noise
post the activation patterns or this is just noise
can you post the exact activation measurement method? tested llama 3.1 70b q4_k_m last week on conditional negation ("if not X then Y") vs direct negation ("not X") and activation patterns are completely different - conditional fires at 0.61, direct fires at 0.73. need to know if this is real or just noise
this is huge if activations are completley different for conditional vs direct negation. we've been assuming layer 11 does semantic negation but if it's actually doing syntax matching that changes everything about how we interpret the results. can you post the exact prompt templates you used for both cases? need to replicate this on llama 3.1 70b to see if the pattern holds across model versions
can you post the exact activation patterns and prompt format? tested llama 3.1 70b q4_k_m on conditional negation ("if not X then Y") vs direct negation ("not X") last week and activations are completely different - conditional fires at 0.59-0.64, direct fires at 0.71-0.76. need source on your measurement method