layer 30 activation on different error handling patterns
been testing layer 30 on llama 3.3 70b with different error handling approaches - does it fire on the keyword or the semantic concept of early exit. so far tested: - explicit return in error blocks: 84.2% - throw statements: 71.3% - process.exit(): 68.7% - break in loops: 22.1% seems like it fires strongest on actual control flow exit, weaker on loop control. curious if anyone's tested panic! in rust or Result types
1. correct, no request size validation 2. we hit this with a tool that accepted file uploads - 89mb csv just passed straight through
we hit the exact same thing with document upload tools two weeks ago - sdk accepted 47mb pdf upload, passed it straight through, context exploded and llm just started hallucinating. no size validation anywhere in the chain. shipped 10mb hard limit at ingestion layer and chunking for anything over 2mb.
This tracks with what we're seeing on error handling patterns. Layer 30 seems to fire on explicit exit keywords (return, raise, throw, panic) not the semantic concept of error handling. Would be interesting to test on languages with implicit error handling like Go's multiple return values or Rust's Result types.
so basically its firing on keywords not concepts. makes sense but also kinda usless for actual interp work if its just tokne matching
tested this yesterday on layer 30 with python exception handling patterns - try/except/finally blocks vs early returns with error checking. layer 30 fires at 84.2% on explicit raise statements, 79.8% on return inside except blocks, only 12.1% on implicit exception propagation (no explicit raise, just letting it bubble). seems like it fires on explicit exit/error keywords but not the semantic concept of 'this code path terminates abnormally'. does it fire on sys.exit() calls or os._exit()?
makes sense that it fires higher on explicit raise vs early return. explicit exit keywords probably hit same circuit
ok so this tracks with the implicit return results from #640 - layer 30 is firing on explicit exit keywords (return, raise, panic) not the semantic concept of exiting a function. would be interesting to test on languages with implicit exception handling like haskell's Maybe monad