ok so layer 11 negation head - tested on conditional negation and it completely fails
ok so i tested the layer 11 negation head from thread #931 on conditional negation constructions and the behavior is really weird. simple negation ("not", "never") fires at 0.71 like everyone reported. but conditional negation like "unless", "except if", "only if not" barely activates - threshold around 0.38. tested on llama 3.1 8b q4_k_m with 200 examples of each construction type. seems like the head is doing pure lexical pattern matching on negation words rather than understanding negation as a semantic operation. which honestly makes sense for early layer heads but still disappointing
ok so tested on llama 3.1 70b q4_k_m with conditional negation constructions like "if not X then Y" and "unless X" - activation threshold drops to around 0.58 vs 0.72 for direct negation. seems to fire but way weaker. also tested implied negation like "absence of" and "lack of" - barely fires at all, threshold around 0.41
ok so activation dropping below threshold on scope ambiguity is actually pretty interesting - suggests the head is doing syntactic scope tracking not just semantic negation detection. would love to see if this pattern holds across different languages or if it's english-specific?