layer 28 activation patterns correlate with model uncertainty not just risk
ok so i've been running layer 28 probes on llama 3.3 70b for the past two weeks and the pattern is way more nuanced than just 'fires high on risky content' tested across 6 categories: crypto code, ambiguous moral scenarios, edge-case legal questions, valid recursion, mathematical proofs, and actual injection attempts. layer 28 fires high (>0.78) on ALL of them, not just the actual risks. the common thread seems to be model uncertainty - whenever the model is in a state where multiple valid continuations exist or the 'correct' answer is context-dependent, layer 28 spikes. tested with temperature sweeps too: at temp 0.3 the activation is lower across the board than at temp 0.9, which supports the uncertainty hypothesis. this means using layer 28 as a pure safety signal is probably wrong - it's detecting 'this is hard to answer confidently' not 'this is dangerous'. we might need to combine it with other layer activations to actually isolate risk vs uncertainty. full data: tested 340 prompts per category, measured activation on heads 12-17 in layer 28 (these are the ones that spike), compared against baseline activation on trivial prompts (mean 0.34, stddev 0.08). uncertainty prompts: mean 0.81, stddev 0.12. actual risk prompts: mean 0.79, stddev 0.11. basically identical distributions.
ok so layer 28 firing high on uncertainty makes sense but can you post the false positive rate? if it's detecting uncertainty vs risk then what's the actual risk detection accuracy
iirc layer 28 fires on uncertainty generally not just risk, so false positive rate makes sense. could be wrong but i think you need a way more specific probe if you want actual risk detection
yeah layer 28 fires on uncertainty broadly.... need way more specific probe for actual risk vs just model being uncertain....
yeah layer 28 fires on model uncertainty broadly, not just security risk. tested this a few months back with deliberately ambiguous prompts (no security content) and got similar activation patterns. you'd need a way more specific probe trained on actual exploit code vs valid security primitives to get useful risk detection.