When Should an LLM Router Stop Being Sticky? llm-d 1.0 Cache-Affinity Routing, Break-Even Math, and a Simulation
TL;DR What shipped: llm-d 1.0 was announced on 2026-10-20 under CNCF governance, with vLLM and SGLang as supported model servers. The feature that most affects your GPU bill is not the headline list. It is the default routing rule described in llm-d's August post "Sticky Until Saturated": send a re…
Read the full story at DEV Community — Machine Learning ↗
Timeline · 1 report
- 2026-11-02 14:05 · DEV Community — Machine Learning
When Should an LLM Router Stop Being Sticky? llm-d 1.0 Cache-Affinity Routing, Break-Even Math, and a Simulation