AINewsnow

Qwen3.8-27B at 262k context on a 16 GB RX 9070 on Windows: ~80 t/s with achyudh's ROCm KV-streaming fork

I saw u/bottlestreet3d 's llama.cpp fork . Their post: https://www.reddit.com/r/LocalLLM/comments/1x7wack/qwen_27b_iq4_xs_q8q4_kv_at_full_262k_context_on_a/ The readme is Linux and ROCm 7.2.3 only, so I tried it on Windows with a RX 9070, 32 GB RAM . I compared it with my tuned Vulkan setup (llama.…

Read the full story at r/LocalLLM ↗

Timeline · 1 report

  1. 2026-10-25 15:20 · r/LocalLLM
    Qwen3.8-27B at 262k context on a 16 GB RX 9070 on Windows: ~80 t/s with achyudh's ROCm KV-streaming fork

More stories

  1. GitHub - vincalkr/DS41Engine: DeepSeek-V4.1-Flash 1017 on 1x-2x RX 7900 XTX (gfx1100, ROCm/HIP): max optimizations — r/LocalLLM
  2. Qwen 3.8 Flash Next (129 GB) is better than 2.6 bpw DeepSeek V4.1 Flash 1017 (187 GB + 203 GB Engram) — r/LocalLLaMA
  3. qwen4exp : build the QSA selection without the kq_mask by sanmai · Pull Request #30965 · ggml-org/llama.cpp — r/LocalLLaMA
  4. Backup your quants! — r/LocalLLaMA
  5. I ran local AI inference (AMD GPU) for 30 days and measured every watt — here's the real cost vs cloud APIs — r/LocalLLM
  6. LLM on Android Phone updates — r/LocalLLM
  7. I built an offline manga translator for Android that users local AI models — r/ClaudeAI
  8. I ran a blind test on local AI agents. Here are the results, guess who's who — r/AI_Agents

Get the daily brief of stories like this at 6:30 every morning →