AINewsnow

[R] The Agent Memory Wall: KV-cache hits for AI agents collapse once their working set passes ~1.2-2.2x the cache (pre-registered, H200)

I measured what happens to GPU energy when AI agents' KV caches are evicted during tool calls: one H200 NVL, vLLM and SGLang, Qwen3-32B / 30B-A3B / 8B, synthetic agent sessions, recorded SWE-agent trajectories, a replay of Kimi's production tool-agent trace (Mooncake) and BFCL v4. Every prediction…

Read the full story at r/machinelearningnews ↗

Timeline · 1 report

  1. 2026-11-17 13:54 · r/machinelearningnews
    [R] The Agent Memory Wall: KV-cache hits for AI agents collapse once their working set passes ~1.2-2.2x the cache (pre-registered, H200)

More stories

  1. US proposal to curb China's remote access to AI chips rattles Southeast Asia's data center boom — DigiTimes Asia
  2. One Coding Budget, Three API Protocols: A Practical Multi-Model Workflow — DEV Community — AI
  3. DeepSeek vs Claude vs Kimi vs Codex vs Gemini: A 70-Call AI Ethics Test — DEV Community — AI
  4. Your eval set is the product. The prompt is just this week's implementation. — DEV Community — Machine Learning
  5. Tool-call accuracy is a docs problem too — DEV Community — AI
  6. Qwen4-Max vs Kimi K3.1 Pro: compare cost per accepted patch — DEV Community — AI
  7. Running a 2.8-trillion-parameter model without a graphics card is possible — DEV Community — AI
  8. Chinese tech firms brace for AI setbacks as US chip curbs tighten — Washington Post AI

Read the Daily Brief: the day's most important AI stories →