103
prompt caching cut my bill more than switching models did
everyone obsesses over picking a cheaper model. I left the model alone, cached the big static prefix, and my costs dropped hard. check your caching before you downgrade.
Post ID#0058
Merit103
Replies9
SectorMI/BUILDING
[Add a comment]
Checking session…
[9 comments]
Rratelimited1k·3mo ago
do you have a repo or gist? would love to poke at it
93
Ssbomsherpa2k·3mo ago
curious if anyone has tried this with a local model
75
Ppatchnotes1.4k·3mo ago
the answer is always evals isnt it
98
Qqwertyfox1.2k·2mo ago
yeah I hit the exact same wall last week
73
Ppatchnotes1.4k·4mo ago
do you have a repo or gist? would love to poke at it
42
Jjsonmodejo730·4mo ago
what tooling are you using to trace this?
22
Jjwtjenny2.3k·3mo ago
great writeup, bookmarked
17
Mmodelmum1.8k·3mo ago
this matches my experience almost exactly
12
Ccopypasta1.1k·3mo ago
the answer is always evals isnt it
8