AINewsnow

Qwen 3.8 27B Q4 with 100K context on a 16 GB RX 7800 XT guide

I'm running Qwen 3.8 27B Q4 XS with ~30 t/s decode (no MTP) at 100k context with q8/q5 KV cache on a 16GB AMD GPU. I wanted to share my setup since many believe Qwen 3.8 27B to be infeasible on 16GB VRAM. Build llama.cpp with Vulkan: cmake -B build -DGGML_VULKAN=ON && cmake --build build --config R…

Read the full story at r/LocalLLaMA ↗

Timeline · 1 report

  1. 2026-09-29 11:18 · r/LocalLLaMA
    Qwen 3.8 27B Q4 with 100K context on a 16 GB RX 7800 XT guide

More stories

  1. Sonnet 5.5 orchestrated a local Qwen 3.8 27B! — r/ClaudeAI
  2. Qwen flash next on 12+16gb vram, and 32gb ram viable? — r/LocalLLM
  3. add GLM-5.3-Flash (GLM5-Next) support by timkhronos · Pull Request #27773 · ggml-org/llama.cpp — r/LocalLLaMA
  4. Used Opus 5.5 to optimize llama.cpp inference for Swift Qwen 3.8 27B Q6_K on RTX 5090 - decode 143 tok/s prefill 2840 tok/s — r/LocalLLM
  5. Qwen 3.8 27B on a single 3090: 114 min solo, 43 min as a worker under a GPT 6.1 SOL orchestrator — r/LocalLLM
  6. Qwen 3.8 27B on a 3090 with a Sonnet 5.5 as a planner: 2.7x cheaper, real numbers — r/LocalLLM
  7. Qwen 3.8 is a workhorse — r/LocalLLaMA
  8. Two open-weights releases: Victoria (Qwen3.8-Flash-Next with 44% of experts cut, 70% Terminal-Bench 2.1, GGUF included) and Maple (a Canada-first fine-tune) — r/LocalLLM

Get the daily brief of stories like this at 6:30 every morning →