claude can now refuse to call a tool if it thinks the args are unsafe right
saw this in the api docs but can't tell if it actually works in practice. has anyone seen the model refuse a tool call because the arguments looked like secrets or injection attempts? or is this just theoretical
yes.... since 3.5 sonnet iirc.... but only if you return a proper error schema in the tool definition
wait it can refuse? i thought it would just call the tool with whatever args and you had to validate server-side
works but only if you pass validate_args: true in tool schema
didn't know about validate_args, is that in the anthropic sdk or langchain
it can refuse but only if you give it good tool descriptions. we added a line like "this tool should never be called with user-provided file paths" and it started refusing way more often. also validate_args helps but i think thats only in anthropic sdk iirc 😊
we added tool descriptions that say "never call this with untrusted input" and it works maybe 60% of the time 😅 the other 40% the model just calls it anyway and we catch it server-side
we're running claude 3.5 sonnet (2024-10-22) with tool validation and it refuses maybe 70% of the time when we mark a tool as requiring trusted input. the other 30% we catch server-side with a regex that blocks anything that looks like a path traversal or command injection. exact numbers from last week: 127 refusals, 51 caught server-side, 3 false positives
we're seeing the same thing on sonnet, it refuses maybe 50% of the time when the tool description includes "user provided" or "untrusted". honestly it's kind of nice because it catches stuff we missed but also breaks legitimate use cases....
wait so claude can actually refuse tool calls now.... that's kind of amazing? does it explain why it refused or does it just silently skip the tool? also what version are you running, i want to test this