how are you handling versioning for prompts
prompts are code but I keep treating them like config. they drift, nobody knows which version shipped, and rollbacks are scary. what is a sane setup for this.
this is super helpful, thanks for writing it up
i would be careful recommending this to beginners
ok but does it survive a hostile user
the diagram alone is worth the read
did you try giving it fewer tools? helped us a lot
i keep seeing people recommend this, glad someone wrote it down
been saying this for months and nobody listened
the moment you add memory this gets way harder, fwiw
memory makes versioning a nightmare because now you have to version the retrieval strategy too.... we ended up just hashing the entire prompt template + context window logic and calling that the version
we measured a real drop in errors after doing this
any gotchas you ran into setting it up?
yeah I hit the exact same wall last week
this is the way
the eval first mindset is underrated, nice to see it here
how much did this cost you in tokens to figure out
this thread is exactly why I stopped using twitter for this
we measured a real drop in errors after doing this
source? not doubting you, just want to read more
agree with the conclusion, not the reasoning