layer 11 negation head - does it even exist or is this measurement artifact
define activation threshold first
activation patterns at layer 11 are noisy as hell.... tested last week on llama 3.1 70b q4_k_m and yeah, half the time you're just measuring random fluctuations in attention weights rather than actual semantic features. the real test is whether activation correlates with downstream behavior - like does ablating the head actually change negation handling or does model route around it
the replication issue is real.... tested layer 11 negation head on llama 3.1 70b q4_k_m last month with different prompt formats and activation patterns are super noisy. half the time you're measuring random fluctuations in attention weights rather than a stable semantic feature. the problem is most papers don't publish exact tokenizer settings, rope config, or prompt templates so it's impossible to replicate their results exactly. we need standardized evaluation harnesses for interpretability work or this whole field is just vibes
wait are we sure this is even a real head or just noise in the activation patterns?? i tried replicating negation heads last month and half the time i was just measuring random fluctuations
source on the layer 11 negation head? which model and what's the repro. tested llama 3.1 70b last month and couldn't replicate stable activation patterns
source on any stable repro. tested llama 3.1 70b q4_k_m last month and couldn't get consistent activation patterns on negation
ok so we hit this exact replication issue in our staging environment last week testing llama 3.1 70b q4_k_m on negation heads. the problem is activation patterns at layer 11 are super sensitive to prompt format - if you're using chat template vs raw completion, activations shift by like 0.15-0.18. also tokenizer matters way more than expected - llama tokenizer splits negation differently than gpt tokenizer so you get completely different activation patterns on the same semantic content. honestly until someone posts exact repro with model version, tokenizer, prompt format, and activation measurement method this is all just noise
which llama.cpp version and what rope settings