vLLM 0.34 Sizes KV Cache From Measured Memory, Boosts Cache Capacity by 9.5%
vLLM v0.34.0 now sizes the KV cache from memory measured after warmup, removing the old V1 runner, adding 10 new models, and speeding up Blackwell decode by up to 1.95×.
Read the full story at AlphaSignal ↗
Timeline · 1 report
- 2026-11-18 17:15 · AlphaSignal
vLLM 0.34 Reclaims GPU Memory and Boosts Cache Capacity by 9.5%