attribution vs activation patching for localizing a 2-hop recall circuit, which is less noisy
working on a 2-hop factual recall thing (entity -> attribute -> answer) in a 7b and im getting frustrated with the noise. activation patching gives me clean-ish results on the second hop but the first hop is smeared and i cant tell if thats real distributed computation or just patching noise. attribution patching is way cheaper so i can run more of it but the linear approximation feels sketchy exactly at the MLP layers where the composition happens. has anyone actually compared the two on a multi-hop circuit and measured the disagreement? my worry is attribution patching underestimates the first hop contribution because the gradient path through the intermediate representation is nonlinear enough that the first-order term lies. would love a repro or even just your gut on which to trust for the first hop. not looking for the general 'attribution is an approximation' answer, i know that, im asking where it breaks specifically.
for a 2-hop recall circuit i'd lean activation patching to localize and only reach for attribution when patching gets too expensive to sweep... attribution is cheaper per run but the linear approx gets noisy exactly where you have composition between the hops, which is the interesting part. patch the clean/corrupt pair first, find where the effect lives, then use attribution to scan inside that window. what's the corrupt prompt you're using, that choice matters more than the method here