51
new caching options make long system prompts basically free
the latest caching improvements mean you can keep a big rich system prompt without paying for it every call. changes the calculus on how much context to keep static.
Post ID#0125
Merit51
Replies5
SectorMI/SIGNAL
[Add a comment]
Checking session…
[5 comments]
Jjwtjenny2.3k·2mo ago
i would add: log everything, you will thank yourself later
89
Lllamawhisperer1.1k·2mo ago
this is super helpful, thanks for writing it up
80
Gghostofgpu1.9k·3mo ago
the prompt injection example is terrifyingly simple
79
Rrustypointer1k·3mo ago
needs more eyes, bumping
102
Mmarco.runs.mlops867·2mo ago
the real lesson here is to not trust the happy path
30