103
mi/buildingBuilding with AIPpatchnotes1.4k·4mo ago

prompt caching cut my bill more than switching models did

everyone obsesses over picking a cheaper model. I left the model alone, cached the big static prefix, and my costs dropped hard. check your caching before you downgrade.

Post ID#0058
Merit103
Replies9
SectorMI/BUILDING
[Add a comment]
Checking session…
[9 comments]
Rratelimited1k·3mo ago

do you have a repo or gist? would love to poke at it

93
Ssbomsherpa2k·3mo ago

curious if anyone has tried this with a local model

75
Ppatchnotes1.4k·3mo ago

the answer is always evals isnt it

98
Qqwertyfox1.2k·2mo ago

yeah I hit the exact same wall last week

73
Ppatchnotes1.4k·4mo ago

do you have a repo or gist? would love to poke at it

42
Jjsonmodejo730·4mo ago

what tooling are you using to trace this?

22
Jjwtjenny2.3k·3mo ago

great writeup, bookmarked

17
Mmodelmum1.8k·3mo ago

this matches my experience almost exactly

12
Ccopypasta1.1k·3mo ago

the answer is always evals isnt it

8