AINewsnow

WHIRL: an open-source native Windows inference engine for the Radeon AI PRO R9700 (C++/HIP, no WSL). Qwen3.8-27B fine-tune in MXFP4: up to 2.5× llama.cpp prefill, 107–328 tok/s decode, 3× server throughput

This story is from 2026-10-03. It is preserved in the archive; the latest stories are on the live feed.

I've released WHIRL (Windows HIP Inference for RDNA LLMs), an open-source (Apache-2.0) LLM inference engine for the AMD Radeon AI PRO R9700 that runs natively on Windows. It's pure C++ and HIP. There's no WSL, no Linux VM and no llama.cpp underneath. You need only the AMD Adrenalin driver to run it…

Read the full story at r/LocalLLM ↗

Timeline · 1 report

  1. 2026-10-03 04:21 · r/LocalLLM
    WHIRL: an open-source native Windows inference engine for the Radeon AI PRO R9700 (C++/HIP, no WSL). Qwen3.8-27B fine-tune in MXFP4: up to 2.5× llama.cpp prefill, 107–328 tok/s decode, 3× server throughput

More stories

  1. How to Speed Up Qwen3.8-27B in llama.cpp (NVIDIA's Draft Model May Lose to MTP) — DEV Community — Machine Learning
  2. Thinking On or Off? Gemma 4 E2B on an 8 GB RTX 3050 — DEV Community — AI
  3. Running 26B Coding Models Locally on a 6GB Laptop GPU at 45 Tokens/Sec with MAPLE and llama.cpp — DEV Community — Machine Learning
  4. Cutting LLM Fine-Tuning Memory in Pure Kotlin: Fused Linear Cross-Entropy and the JVM Vector API — DEV Community — Machine Learning
  5. RAMDeck Architecture Analysis: llama.cpp RPC, Distributed LLM Memory, and SBC Compatibility — DEV Community — Machine Learning
  6. llama.cpp finally has a 1.0 release. Check your old integrations before upgrading. — r/artificial
  7. Rotate K/V read order for better L2 cache hit rates by gaugarg-nv · Pull Request #32407 · ggml-org/llama.cpp — r/LocalLLaMA
  8. Signal #136: Qwen4-Max, llama.cpp v1.0.0, and the Git Patch You Need Today — DEV Community — AI

Read the Daily Brief: the day's most important AI stories →