What a 70% vs 90% prompt cache hit rate does to an AI agent’s bill (72 models priced)
An agent sends the same long prefix on every step: system prompt, tool definitions, the files it has read, the conversation so far. Most of it is already in the provider's prompt cache from the step before. How much of it is, the cache hit rate, ends up deciding most of the bill. I wanted a number…
Read the full story at DEV Community — AI ↗
Timeline · 1 report
- 2026-11-09 14:37 · DEV Community — AI
What a 70% vs 90% prompt cache hit rate does to an AI agent’s bill (72 models priced)