▲ NVIDIA shows 3 ways to speed up MoE inference on DGX Spark, tested on Ant Group inclusionAI's Ling-3.0-flash (+42.5% decode throughput on GSM8K)
This story is from 2026-11-03. It is preserved in the archive; the latest stories are on the live feed.
NVIDIA published a technical blog profiling Ling-3.0-flash-fp4 (124B total, 5.1B active, MIT) on a single DGX Spark at batch size 1. Decoding was memory-bound, so they stacked 3 optimizations: Humming MXFP4 MoE kernels, an online FP8 LM head and MTP speculative decoding using the model's built-in N…
Read the full story at r/machinelearningnews ↗
Timeline · 1 report
- 2026-11-03 20:40 · r/machinelearningnews
▲ NVIDIA shows 3 ways to speed up MoE inference on DGX Spark, tested on Ant Group inclusionAI's Ling-3.0-flash (+42.5% decode throughput on GSM8K)