AINewsnow

SageMaker AI adds KV cache-aware routing and cross-Region inference routing

This story is from 2026-11-17. It is preserved in the archive; the latest stories are on the live feed.

Amazon SageMaker AI real-time endpoints now support KV cache-aware routing, which sends requests to instances with the longest cached prompt match. AWS also detailed cross-Region inference routing, used by Salesforce to serve models across 30+ Regions and hundreds of accounts.

Read the full story at AWS Machine Learning Blog ↗

Timeline · 4 reports

  1. 2026-11-19 18:20 · AWS Machine Learning Blog
    Keep inference running through regional capacity shortfalls with cross-Region inference routing in Amazon SageMaker AI
  2. 2026-11-19 09:58 · DEV Community — Machine Learning
    How Salesforce uses cross-Region inference routing to address model-serving capacity gaps
  3. 2026-11-18 20:13 · AWS Machine Learning Blog
    How Salesforce is advancing global-scale model serving with cross-Region inference routing in Amazon SageMaker AI
  4. 2026-11-17 21:11 · AWS Machine Learning Blog
    Amazon SageMaker AI real-time endpoints now support KV cache aware routing

More stories

  1. Anthropic debuts on Nasdaq in record $81.3 billion IPO, shares soar 21% — Anthropic (X)
  2. AWS Adds AI Data Preparation Capabilities to Amazon SageMaker — AWS Machine Learning Blog
  3. AWS expands SageMaker JumpStart with OpenTSLM, model leaderboard, and PolyAI Dialog-RSN-1 — AWS Machine Learning Blog
  4. AWS Adds Managed NVIDIA Dynamo Support to SageMaker HyperPod Inference — AWS Machine Learning Blog
  5. AWS expands Bedrock AgentCore with OAuth, networking, and cost-control patterns — AWS Machine Learning Blog
  6. Amazon and Alphabet to Hold Over 28% of Anthropic After Its IPO — The Globe and Mail
  7. Satya Nadella says AI agents will reshuffle the economics of the web — TechCrunch AI
  8. Disney and AWS unveil AI-powered Star Wars audio-animatronics — CNET AI

Read the Daily Brief: the day's most important AI stories →