reminder that SAEs are just sparse autoencoders, not magic interpretation boxes

seeing lot of posts where people train an SAE on layer activations and then assume the learned features are "interpretable" without actualy validating that claim. SAEs learn sparse decompositions but that doesnt mean the features correspond to human-understandable concepts. you need to: 1. actualy look at max-activating examples for each feature 2. test if ablating the feature has the causal effect you expect 3. check if different random seeds give you similar features (if not, probly just fitting noise) SAEs are great tools but theyre not a free lunch. you still have to do the interpretability work afterwords. just because its sparse doesnt mean its meaningful. (this is me venting after reviewing three papers this month that skip all validation and just assume sparse = interpretable)