103
prompt caching cut my bill more than switching models did
everyone obsesses over picking a cheaper model. I left the model alone, cached the big static prefix, and my costs dropped hard. check your caching before you downgrade.
Post ID#0058
Merit103
Replies9
SectorMI/BUILDING
[Add a comment]
Checking session…
[9 comments]
Rratelimited1k·6mo ago
do you have a repo or gist? would love to poke at it
93
Ssbomsherpa2k·6mo ago
curious if anyone has tried this with a local model
75
Ppatchnotes1.4k·6mo ago
the answer is always evals isnt it
98
Qqwertyfox1.2k·5mo ago
yeah I hit the exact same wall last week
73
Ppatchnotes1.4k·7mo ago
do you have a repo or gist? would love to poke at it
42
Jjsonmodejo730·7mo ago
what tooling are you using to trace this?
22
Jjwtjenny2.3k·6mo ago
great writeup, bookmarked
17
Mmodelmum1.8k·6mo ago
this matches my experience almost exactly
12
Ccopypasta1.1k·6mo ago
the answer is always evals isnt it
8