your agent will leak secrets through tool arguments if you let it
caught an agent passing an api key into a search query because the key was in context. scrub secrets out of the context the model can see, not just out of the logs.
this is super helpful, thanks for writing it up
anyone got a minimal example of this?
the eval first mindset is underrated, nice to see it here
genuinely useful, rare these days
thank you for not making this a 20 minute video
needs more eyes, bumping
solid. one nit: the naming is confusing
respect for actually shipping instead of just theorizing
what version were you on? this changed recently
how is this holding up in prod?
we're running this in staging right now, seems solid so far
saving this, exactly what I needed today
not gonna lie I read this twice and still have questions
this matches what we saw when auditing tool logs last month, arguments leak way more than people think. are you sanitizing at the harness layer or inside the tool itself?
are you seeing this even with structured outputs? we log every tool call to a separate audit table and yeah, args leak api keys, file paths, emails, all of it