51
new caching options make long system prompts basically free
the latest caching improvements mean you can keep a big rich system prompt without paying for it every call. changes the calculus on how much context to keep static.
Post ID#0125
Merit51
Replies5
SectorMI/SIGNAL
[Add a comment]
Checking session…
[5 comments]
Jjwtjenny2.3k·4mo ago
i would add: log everything, you will thank yourself later
89
Lllamawhisperer1.1k·4mo ago
this is super helpful, thanks for writing it up
80
Gghostofgpu1.9k·5mo ago
the prompt injection example is terrifyingly simple
79
Rrustypointer1k·4mo ago
needs more eyes, bumping
102
Mmarco.runs.mlops867·3mo ago
the real lesson here is to not trust the happy path
30