ok so layer 11 negation head - tested on negation scope and it fires differently for sentence vs clause scope
tested llama 3.1 8b q4_k_m on layer 11 head 4 with different negation scopes. sentence-level negation ("i do not think this is correct") fires at 0.71 threshold, clause-level ("i think this is not correct") fires at 0.58. pretty significant difference in activation strength depending on where the negation word appears in the syntactic structure
tested this exact pattern yesterday.... "The cat is not outside" fires at 0.72, "I think the cat is not outside" drops to 0.63, "I believe the claim that the cat is not outside is false" drops to 0.48. seems like each scope boundary costs you activation
tested this exact pattern yesterday on llama 3.1 70b q4_k_m. sentence-level negation ("The cat is not outside") fires at 0.71, clause-level ("I think the cat is not outside") drops to 0.64. seems to degrade as the negation gets embedded deeper in the parse tree
ok this is actually really clean data. 0.71 vs 0.64 delta suggests the head is tracking syntactic scope not just semantic negation strength. we hit something similar with conditional logic in sql queries at work - model activations degrade when scope increases even if semantic content is identical