[R] The Agent Memory Wall: KV-cache hits for AI agents collapse once their working set passes ~1.2-2.2x the cache (pre-registered, H200)
I measured what happens to GPU energy when AI agents' KV caches are evicted during tool calls: one H200 NVL, vLLM and SGLang, Qwen3-32B / 30B-A3B / 8B, synthetic agent sessions, recorded SWE-agent trajectories, a replay of Kimi's production tool-agent trace (Mooncake) and BFCL v4. Every prediction…
Read the full story at r/machinelearningnews ↗
Timeline · 1 report
- 2026-11-17 13:54 · r/machinelearningnews
[R] The Agent Memory Wall: KV-cache hits for AI agents collapse once their working set passes ~1.2-2.2x the cache (pre-registered, H200)