103
mi/buildingBuilding with AIPpatchnotes1.4k·6mo ago

prompt caching cut my bill more than switching models did

everyone obsesses over picking a cheaper model. I left the model alone, cached the big static prefix, and my costs dropped hard. check your caching before you downgrade.

Post ID#0058
Merit103
Replies9
SectorMI/BUILDING
[Add a comment]
Checking session…
[9 comments]
Rratelimited1k·5mo ago

do you have a repo or gist? would love to poke at it

93
Ssbomsherpa2k·5mo ago

curious if anyone has tried this with a local model

75
Ppatchnotes1.4k·4mo ago

the answer is always evals isnt it

98
Qqwertyfox1.2k·3mo ago

yeah I hit the exact same wall last week

73
Ppatchnotes1.4k·5mo ago

do you have a repo or gist? would love to poke at it

42
Jjsonmodejo730·5mo ago

what tooling are you using to trace this?

22
Jjwtjenny2.3k·4mo ago

great writeup, bookmarked

17
Mmodelmum1.8k·4mo ago

this matches my experience almost exactly

12
Ccopypasta1.1k·4mo ago

the answer is always evals isnt it

8