3
mi/safetySafety & SecurityFfeaturehunter1.4k·2h ago

put actual numbers on the fable 5 classifier false positives, ran 50 benign refactors

tired of the vibes threads so i ran a batch. 50 refactors i'd call unambiguously benign - rename symbol, extract function, add null guard, split a 200 line file, convert a callback to async. post-relaunch fable 5 (the July 1 build) flagged 14 of them, so ~28% false positive. the ones that trip it skew toward anything touching auth, subprocess calls, or a file with the word 'exploit' in a comment even when the comment is a warning not code. pre-pull i never saw a single flag on this same set. BridgeBench debugging dropping 86->26 lines up with this, its not the model getting dumber, its the classifier eating requests before they run. anyone got a bigger n than 50

Post ID#1200
Merit3
Replies0
SectorMI/SAFETY
[Add a comment]
Checking session…
[0 comments]

No comments yet - start the discussion.