AINewsnow

[GUIDE] Qwen3.8 Flash-Next on a single RTX 5080 + 64GB RAM: >250 tok/s tg and 5700 tok/s prefill in real agent coding sessions (Basalt + new Unsloth UD-IQ3_XXS + Lamina drafter),+ how it compares to llama.cpp

This story is from 2026-11-05. It is preserved in the archive; the latest stories are on the live feed.

TL;DR Basalt is the engine and it's doing the heavy lifting. Same model file, same box: about 3.5x llama.cpp's decode without drafting and ~5x with it, and ~7x faster prompt reads . Unsloth's new UD-IQ3_XXS (Dynamic v3.1, uploaded Nov 4) is the same size as the GSQ-RCO IQ3_S everyone uses on 16GB c…

Read the full story at r/LocalLLaMA ↗

Timeline · 1 report

  1. 2026-11-05 16:23 · r/LocalLLaMA
    [GUIDE] Qwen3.8 Flash-Next on a single RTX 5080 + 64GB RAM: >250 tok/s tg and 5700 tok/s prefill in real agent coding sessions (Basalt + new Unsloth UD-IQ3_XXS + Lamina drafter),+ how it compares to llama.cpp

More stories

  1. Probably old news, but any website you have open can reach your local LLM server while you browse — r/LocalLLM
  2. The Open-Source Paradox: Giving Away AI Makes It Safer (and More Dangerous) — DEV Community — AI
  3. Llama-modchip: 284B DeepSeek on 64GB RAM, no GPU — Hacker News Front Page
  4. CyberTiel35BA3B working on Blender Rigging via MCP. 8GB VRAM, 16GB RAM. llama.cpp. — r/LocalLLaMA
  5. Zero-Downtime LLM Inference on Kubernetes Spot Instances (~70% GPU Cost Savings) — DEV Community — Machine Learning
  6. The inference engine flood.. but where's the quality? — r/LocalLLM
  7. moe-cache : LRFU/MQ-like hybrid cache, ClassLRU by sanmai · Pull Request #31233 · ggml-org/llama.cpp — r/LocalLLaMA
  8. Opus 5.5 wrote its own quantiser for Qwen3.5-0.8B, no llama-quantize, no imatrix. 5.9× better than Unsloth's same-size file. — r/LocalLLM

Get the daily brief of stories like this at 6:30 every morning →