Your chatbot resends the whole conversation on every message. Here's what that costs, measured.
This story is from 2026-10-15. It is preserved in the archive; the latest stories are on the live feed.
Chat APIs are stateless. To answer message 41, your app sends messages 1 to 40 again, and you pay for all of them as input tokens, every single time . On our own API traffic over the last 7 days (about 411,000 requests), 99.3% of all tokens processed were input tokens . The median request sent 3,78…
Read the full story at DEV Community — AI ↗
Timeline · 1 report
- 2026-10-15 15:01 · DEV Community — AI
Your chatbot resends the whole conversation on every message. Here's what that costs, measured.