AINewsnow

Google Cloud Ships llm-d 1.0 With Cache-Aware Routing for TPU and GPU Fleets

Google, Red Hat and IBM released llm-d 1.0, an open-source Kubernetes-native stack for distributed LLM inference. It adds multi-tenant flow control, zero-downtime upgrades and agent-aware routing across vLLM, SGLang, TPUs and GPUs. The post Google Cloud Ships llm-d 1.0 With Cache-Aware Routing for…

Read the full story at MarkTechPost ↗

Timeline · 1 report

  1. 2026-10-21 04:33 · MarkTechPost
    Google Cloud Ships llm-d 1.0 With Cache-Aware Routing for TPU and GPU Fleets

More stories

  1. Gemini on Android Auto can access my photos, emails, and calendar now – why I’m OK with that (mostly) — ZDNET AI
  2. Ex-DeepMind researcher Thore Graepel launches new startup Metis Reasoning — Sifted
  3. Introducing TorchTPU — PyTorch Blog
  4. New Gemini tools for small business owners — Google Gemini Blog
  5. Google open-sources Harness²: agents that rewrite their own harness on unseen tasks, with frozen weights and no labels — r/machinelearningnews
  6. Welcome to Google Gemini 4 Argon — Towards Data Science
  7. Introducing Gemini for government services — Google Cloud AI Blog
  8. Developers can now access SageMaker Studio through the new AWS experience — AWS Machine Learning Blog

Get the daily brief of stories like this at 6:30 every morning →