How to Speed Up Qwen3.8-27B in llama.cpp (NVIDIA's Draft Model May Lose to MTP)
NVIDIA just shipped a speculative decoding draft model for Qwen3.8-27B, and the model card doesn't contain a single tokens-per-second number. It has acceptance figures, a comparison against Qwen's built-in MTP head, and a note that it was tested on a Groq 3 LPU. If you run the 27B on a 24 GB consum…
Read the full story at DEV Community — Machine Learning ↗
Timeline · 1 report
- 2026-11-19 17:10 · DEV Community — Machine Learning
How to Speed Up Qwen3.8-27B in llama.cpp (NVIDIA's Draft Model May Lose to MTP)