AINewsnow

How to Speed Up Qwen3.8-27B in llama.cpp (NVIDIA's Draft Model May Lose to MTP)

NVIDIA just shipped a speculative decoding draft model for Qwen3.8-27B, and the model card doesn't contain a single tokens-per-second number. It has acceptance figures, a comparison against Qwen's built-in MTP head, and a note that it was tested on a Groq 3 LPU. If you run the 27B on a 24 GB consum…

Read the full story at DEV Community — Machine Learning ↗

Timeline · 1 report

  1. 2026-11-19 17:10 · DEV Community — Machine Learning
    How to Speed Up Qwen3.8-27B in llama.cpp (NVIDIA's Draft Model May Lose to MTP)

More stories

  1. Exact VRAM & Memory Bandwidth Math for Running DeepSeek-R1, Qwen-2.5-Coder-32B & Llama-3.3-70B Locally — DEV Community — AI
  2. Rotate K/V read order for better L2 cache hit rates by gaugarg-nv · Pull Request #32407 · ggml-org/llama.cpp — r/LocalLLaMA
  3. Qwen4-Max Lands in API, llama.cpp Hits 1.0.0, and 80B Models Run on Mobile Devices — DEV Community — Machine Learning
  4. US proposal to curb China's remote access to AI chips rattles Southeast Asia's data center boom — Ars Technica AI
  5. Nvidia Forms Board Committee on Security and Export Controls After US Chip Curbs — Bloomberg AI
  6. Alibaba CEO Says U.S. Rule Blocking Nvidia AI Chips Will Affect Next Year's Products — The Information AI
  7. Homelab builder runs Qwen 3.6 35B on three used Nvidia V100s at 100 tokens per second — DEV Community — Machine Learning
  8. Thinking On or Off? Gemma 4 E2B on an 8 GB RTX 3050 — DEV Community — AI

Read the Daily Brief: the day's most important AI stories →