AINewsnow

LLM agent token cost: why a kept chat grows with the square of the steps, and the measured fix (prompt caching counted)

An agent that keeps its whole chat resends everything on every step, so the tokens it sends grow with the square of the number of steps. Holding the state on the host and showing the model the exact state keeps the prompt the size of the state. Measured on Gemini 3.8 Flash: 85 times fewer prompt to…

Read the full story at DEV Community — AI ↗

Timeline · 1 report

  1. 2026-11-18 05:08 · DEV Community — AI
    LLM agent token cost: why a kept chat grows with the square of the steps, and the measured fix (prompt caching counted)

More stories

  1. OpenAI launches GPT-6.2 Astra and Sol as Google ships Gemini 4 Neon and Microsoft unveils MAI Code 2 — OpenAI News
  2. Google launches Gemini 4 Neon and opens Genie 4 world model to Gemini API developers — Google DeepMind YouTube
  3. ChatGPT, Claude, Gemini and Grok helped plan terror attacks in US government test — The Independent Tech
  4. Google Adds Computer Use to Gemini App for Mac — Techmeme
  5. Google Introduces Tool Retrieval in Preview for Gemini API — Google: The Keyword
  6. Developers increasingly run Claude Code alongside rival coding agents and alternative backends — DEV Community — AI
  7. Google DeepMind's Koray Kavukcuoglu discusses Gemini and the path to AGI in new podcast episode — Google DeepMind: The Podcast
  8. Google releases Gemini 4 Neon, slotting between Hydrogen and Carbon in its line-up — The New Stack AI

Read the Daily Brief: the day's most important AI stories →