AINewsnow

Qwen3.8-27B Jetson Thor: 40 tok/s at FP16-level accuracy (INT4 + MTP + FlashHead)

I'm on the team at Embedl that built this, and wanted to share the numbers. What's in it INT4 weights, using our own quantization, about half the error of standard AWQ FlashHead, our faster LM head: the 262k-vocab output layer is a real cost at batch 1, so it only scores the likely candidates MTP h…

Read the full story at r/LocalLLaMA ↗

Timeline · 2 reports

  1. 2026-10-26 18:00 · r/LocalLLM
    Qwen3.8-27B Jetson Thor: 40 tok/s at FP16-level accuracy (INT4 + MTP + FlashHead)
  2. 2026-10-26 17:56 · r/LocalLLaMA
    Qwen3.8-27B Jetson Thor: 40 tok/s at FP16-level accuracy (INT4 + MTP + FlashHead)

More stories

  1. Meta announces $95B offering to fund AI — Semafor Technology
  2. Introducing GPT-6.1 Terra — OpenAI News
  3. OpenAI announces three breakthroughs in computer science, math, and physics, including that the Exponential Time Hypothesis, a longstanding conjecture, is false (OpenAI) — Techmeme
  4. Anthropic’s IPO filing lays out a plan to ‘revolutionize every area of human endeavor' with AI — The Verge AI
  5. Microsoft offers buyouts to 60,000 employees — The Verge AI
  6. Anthropic's Claude bypassed a paywall to access Georgia voter data without authorization — Axios AI+
  7. Amp puts AI agents to work beyond code with OpenAI models — OpenAI YouTube
  8. Anthropic's IPO: What Its S-1 Reveals About The AI Boom — Forbes AI

Get the daily brief of stories like this at 6:30 every morning →