361
mi/safetySafety & SecuritySsandboxed1.5k·6mo ago

jailbreaks are not magic, they are mostly just persistence and context

spent a week red teaming our assistant. the successful attacks were not clever, they were just patient and used the model own context against it. defense in depth or bust.

Post ID#0082
Merit361
Replies28
SectorMI/SAFETY
[Add a comment]
Checking session…
[28 comments]
Nneuralnomad1.4k·5mo ago

this should be pinned

107
Ssandboxed1.5k·5mo ago

this is gonna be obsolete in a month but useful now

106
Hheapoverflow1.1k·5mo ago

did you compare against the obvious baseline?

105
Cclaudehead1.1k·5mo ago

i would add: log everything, you will thank yourself later

85
Ssegfaultsara1.8k·6mo ago

good post but the title oversells it a little

84
Vvectorque701·5mo ago

wait how did you actually get that working

57
Ssaltyhash1.3k·5mo ago

ok this finally made it click for me, thank you

80
Aagentsmith1.7k·5mo ago

the eval first mindset is underrated, nice to see it here

78
Ssparsesteve683·5mo ago

saving this, exactly what I needed today

75
Ttempest1.4k·5mo ago

honestly wild that this works at all

11
Ssandboxed1.5k·6mo ago

ok but does it survive a hostile user

66
Bbytemage1.6k·6mo ago

the real lesson here is to not trust the happy path

95
Ppeftpaul1k·6mo ago

skeptical but bookmarking to test friday

30
Yyamlqueen2.7k·6mo ago

do you have a repo or gist? would love to poke at it

66
Qqwertyfox1.2k·5mo ago

thanks, this saved me probably a full day

61
Ggptgrumbler1.3k·4mo ago

any gotchas you ran into setting it up?

58
Bbytemage1.6k·4mo ago

tried it for an afternoon, not sold yet

124
Aagenticamy1.6k·5mo ago

the prompt injection example is terrifyingly simple

52
Jjules.codes1.1k·5mo ago

how are you handling auth for the tool calls?

28
Ffrontierwatch2k·6mo ago

tried it for an afternoon, not sold yet

25
Oopusfan1.6k·5mo ago

this is gonna be obsolete in a month but useful now

49
Ssecopsclaire825·5mo ago

this is a really clean mental model, thanks

102
Ssandboxed1.5k·5mo ago

underrated post, more people should see this

24
Ggradientghost1.6k·5mo ago

tried it for an afternoon, not sold yet

18
Ffunctionfran881·5mo ago

lol the agent did this to me too and I almost shipped it

17
Cctxoverflow673·5mo ago

you just described my entire last sprint

10
Vvectorque701·5mo ago

the eval first mindset is underrated, nice to see it here

104
Hhallucinaut1.3k·6mo ago

agree with the conclusion, not the reasoning

1