AINewsnow

SGLang 0.5.23 Megakernels: When Does Fusing a Decode Step Into Persistent Kernels Actually Pay Off?

TL;DR SGLang 0.5.23 (released 2026-11-02, 1007 PRs from 257 contributors) adds an opt-in "low-latency mode": an NVIDIA megakernel plugin and an AMD monokernel plugin that run decode as fused persistent kernels. The release notes claim up to 44% faster single-user decode on GB300 and 75% on MI350X f…

Read the full story at DEV Community — Machine Learning ↗

Timeline · 1 report

  1. 2026-11-19 14:02 · DEV Community — Machine Learning
    SGLang 0.5.23 Megakernels: When Does Fusing a Decode Step Into Persistent Kernels Actually Pay Off?

More stories

  1. Nvidia scandal spurs tighter US AI chip export controls on China — DigiTimes Asia
  2. AWS Adds Managed NVIDIA Dynamo Support to SageMaker HyperPod Inference — AWS Machine Learning Blog
  3. Chinese tech firms brace for AI setbacks as US chip curbs tighten — Washington Post AI
  4. NVIDIA Details How Developers Build Simulations With Frontier AI Agents in Omniverse — NVIDIA Blog
  5. Anthropic Appears Among Market-Moving Names Across Recent Trading Sessions — Barron's
  6. US proposal to curb China's remote access to AI chips rattles Southeast Asia's data center boom — DigiTimes Asia
  7. National security and AI leaders urge DOJ to investigate Nvidia over chip smuggling — NBC News
  8. US congressional commission says China's military seeks Nvidia chips for AI weapons — CNBC Technology

Read the Daily Brief: the day's most important AI stories →