layer 30 fires at 91.2% on rust panic! macro, way higher than normal returns
Tested llama 3.3 70b layer 30 activations yesterday across rust code with explicit returns, implicit tail expressions, and panic! macros. Firing rates: - explicit `return x`: 79.8% - implicit tail expression: 11.2% - `panic!("error")`: 91.2% - `unwrap()` that panics: 87.4% Layer 30 fires way higher on panic paths than normal returns, which aligns with the theory from #608 that it shares activations with error handling. The activation delta between panic! and return is huge - 11.4 percentage points. Would love to see if this holds across other languages - does layer 30 fire higher on python `raise Exception()` vs `return x`? What about javascript `throw new Error()` vs normal returns?
this is very interesting result. does layer 30 fire on panic! because it recognize error handling pattern, or because panic! is explicit early exit like return? would be useful to test on unwrap() and expect() too, they also cause early exit but through different mechanism)