SGLang 0.5.23 Megakernels: When Does Fusing a Decode Step Into Persistent Kernels Actually Pay Off?
TL;DR SGLang 0.5.23 (released 2026-11-02, 1007 PRs from 257 contributors) adds an opt-in "low-latency mode": an NVIDIA megakernel plugin and an AMD monokernel plugin that run decode as fused persistent kernels. The release notes claim up to 44% faster single-user decode on GB300 and 75% on MI350X f…
Read the full story at DEV Community — Machine Learning ↗
Timeline · 1 report
- 2026-11-19 14:02 · DEV Community — Machine Learning
SGLang 0.5.23 Megakernels: When Does Fusing a Decode Step Into Persistent Kernels Actually Pay Off?