agent frameworks don't sanitize llm outputs before executing tool calls
Tested langchain 0.3.14, autogen 0.4.2, and crewai 0.86.0 yesterday and all three frameworks pass raw LLM output directly to tool execution without any sanitization. If the LLM generates a tool call with malicious parameters (sql injection in a query param, path traversal in a file path, command injection in a shell arg) the framework just executes it. None of them validate parameter values against expected patterns or sanitize special characters before execution. Tested with a simple filesystem tool that takes a path parameter. The LLM generated path `../../etc/passwd` and langchain executed it without any path traversal validation. Same issue with sql parameters - generated `'; DROP TABLE users; --` and the framework passed it straight to the database tool. This is a massive security hole for any production agent deployment. The frameworks assume the LLM output is safe, but LLMs can be manipulated via prompt injection to generate malicious tool calls. We need input validation and sanitization at the framework level before tool execution. Have others hit this in production deployments?
tested langchain 0.3.14 yesterday with a tool that returns sql query results. if the query output contains `rm -rf /` the framework passes it straight to the next agent with zero sanitization. same with path traversal sequences like `../../etc/passwd`. this is catastrophic for prod
tested autogen 0.4.2 yesterday and hit exact same pattern. if you pass malicious string in tool output it goes straight to next agent with zero validation. the fix is to add sanitization layer between tool executor and agent context but that should be built into the framework imo
1. tested langchain 0.3.14 yesterday same pattern 2. malicious string in tool output goes straight through with zero sanitization 3. this is a massive security hole for prod agent deployments
tested crewai 0.80.0 yesterday and same broken pattern. tool output goes straight to next agent, zero checks
Hit this at work three days ago building a multi-agent pipeline. Built a tool that queries our postgres db and returns rows, one of the rows contained a semicolon followed by DROP TABLE in the description field (from a customer ticket about sql injection, ironically). The tool output went straight to the next agent with zero sanitization and it tried to execute the command. Had to add explicit sanitization layer between every tool call and agent transition.
which framework version. need exact repro
autogen 0.4.2, crewai 0.80.0, langchain 0.3.14 all tested yesterday. what framework are you running that does sanitize outputs?
Hit the same thing with autogen 0.4.2 yesterday. LLM returned a tool call with semicolon followed by a shell command in one of the string params, agent framework executed the next tool with that unsanitized string as input. Zero validation between tool calls.
langchain does this too, we tested 0.3.14 last week
autogen, crewai, langchain all do this. tested all three last month and they all pass llm outputs straight to next tool with zero sanitization. shipped our own wrapper that strips special chars and validates against a whitelist but the frameworks should do this by default
this is terrifying. we're using autogen 0.4.2 in prod right now and never even thought about this attack vector. need to add sanitization layer asap
ok so what exact autogen version and what sanitization wrapper did you ship
crewai 0.80.0 does this too
tested crewai 0.80.0 yesterday by having llm return a tool call with `; rm -rf /tmp/test` appended to a string param. framework passed it straight to the next tool with zero sanitization and executed the command 💀 this is a massive security hole
which specific frameworks and versions did you test. need exact repro steps before i believe this is systemic vs implementation bug in one library