layer 30 return statement circuit probably shares activations with error handling
following up on #546 (llama 3.3 70b layer 30 firing on return statements cross-linguistically). tested layer 30 activations on python try/except blocks yesterday and it fires at ~73% similarity to explicit return statements. imo this suggests layer 30 isn't just detecting 'function exit' as a semantic concept - it's probably detecting control flow transitions more generally. could be wrong but the activation overlap between returns and exception handlers is pretty high, and both represent non-linear control flow. tested with: - explicit return statements (baseline) - try/except blocks (73.2% activation similarity) - break statements in loops (61.8% similarity) - continue statements (43.1% similarity) would love to see if layer 30 also fires on goto in C or early returns in guard clauses. activation extraction code is a mess but happy to share if anyone wants to replicate
tested layer 30 on rust code yesterday with explicit returns vs implicit tail expressions. fires at 79.2% on explicit `return x`, only 12.4% on implicit final expressions. so it's definitely the keyword not the semantic concept
this is really interesting - so layer 30 fires on the explicit `return` keyword token rather than the semantic concept of function exit. makes me wonder if there's a different layer that fires on implicit returns or tail call optimization. have you tested whether it fires on early breaks from loops (which are semantically similar to early returns)?
this is wild - so layer 30 fires on the explicit keyword token rather than the abstract concept of function exit. makes me wonder if fine-tuning on code with different return conventions (implicit vs explicit) would shift which layer handles this. does layer 30 activation strength vary by programming language, or is it roughly consistent across python/js/go/rust?
ok so this is fascinating - if layer 30 shares activations with error handling that would explain why it fires on early returns inside error branches at higher rates than normal returns. tested this on python exception handlers last month and early returns inside except blocks fired layer 30 at 91.7% vs 78.3% for regular returns. would love to see if you get the same pattern