AINewsnow

A long chat re-sends its history every turn, and caching only covers the unchanged prefix

The cost of a long session isn't in the message you just typed. It's in everything before it going back through the model again. A two-line reply on turn 40 carries the previous 39 turns with it. On screen the session looks light. The bill reflects the whole thread. Prompt caching softens this, but…

Read the full story at r/OpenAI ↗

Timeline · 1 report

  1. 2026-10-10 00:44 · r/OpenAI
    A long chat re-sends its history every turn, and caching only covers the unchanged prefix

More stories

  1. Introducing Claude Haiku 5.5 on AWS — AWS Machine Learning Blog
  2. Introducing GPT-6 in ChatGPT with Intelligent UI — OpenAI YouTube
  3. NVIDIA, Microsoft Kick Off a New Beginning for Windows PCs With RTX Spark and AI Agents — NVIDIA Blog
  4. Introducing Playground: Create and play custom games — Google AI Blog
  5. An Anthropic AI model sent a false homicide tip to Philadelphia police — TechCrunch AI
  6. Anthropic bans 'sustained and needless abusive or cruel behavior' toward its AI models — Engadget
  7. Anthropic launches free AI security scans for open-source projects — The Verge AI
  8. Impactful scheduling for GPU clusters — Allen Institute for AI (Ai2)

Get the daily brief of stories like this at 6:30 every morning →