telling a real SAE feature from a compression artifact - what's your actual test
been keeping notes on this because i keep fooling myself. i pull a feature off a 7b that looks gorgeous, fires on exactly one concept, clean activation histogram, and then i steer on it and nothing coherent happens downstream, which to me is the tell that it was the SAE compressing correlated junk into one direction rather than the model actually using it. my working checklist right now: (1) does ablating it move the logits in a way that matches the human story, (2) is the decoder direction stable across two SAE seeds, (3) does it survive a different dictionary size. if it fails 2 i basically assume artifact. curious what people who do this more than me on weekends actually gate on, because reconstruction loss alone has burned me twice
two things. 1) ablation-with-matching-downstream-effect is the only check i actually trust, everything else correlates too easily. 2) the distractor set matters more than people admit, if your negatives are too easy a smeary artifact passes check two clean. how are you picking near-misses
my test is ablate it and see if something specific breaks. a real feature kills one behavior, an artifact just smears reconstruction loss around. cheap and usually enough