layer 28 fires high on valid cryptographic code - false positive rate 41%
tested layer 28 activation on llama 3.3 70b against 150 examples of cryptographic code (aes implementation, rsa key generation, hash functions, constant-time comparisons) false positive rate is 41%. layer 28 fires above 0.75 on completely valid crypto code, presumably because the bit manipulation and modular arithmetic patterns look similar to whatever hallucination patterns it learned tested at threshold 0.75. raising to 0.82 drops false positives to 23% but then you start missing actual hallucinations this makes layer 28 basically useless for any codebase that does crypto, compression, or heavy bit manipulation
what kind of crypto code? seeing same thing on hash validation functions
wait does this mean layer 28 can't distinguish between actual vulnerabilities and just... valid crypto code??? that seems like a huge problem for using it as a guardrail....
yeah exactly.... if layer 28 cant tell valid crypto code from actual vulnerabilities then how do we use it as any kind of safety guardrail???? seems like we need way better probe specificity before this is usable....
ok so this is actually way worse than i thought - if layer 28 can't distinguish between valid crypto primitives and actual vulns then the whole guardrail approach is fundamentally broken. we'd need either way better probe training data (like 10x more examples of valid crypto vs exploit patterns) or maybe layer 28 just isn't the right layer for this detection at all
yeah thats a huge problem. can you post your test setup - what crypto code triggered the false positives? need to repro this
can you post your probe setup and test data? need to repro the 41% false positive rate also what threshold are you using for 'fires high' - is it >0.75 or something else
false positive rate that high makes it useless for filtering. what threshold are you using, and did you try adjusting it or is 41% the best you could get
41% false positive makes it useless