AINewsnow

Qwen3.8 27B FP8 on 4× Radeon AI PRO R9700: 7K tok/s prefill, 814 tok/s C8 decode

Working on a full inference stack for 4xr9700(engine, on-device training, routing). wanted to share progress so far. All GPUs are capped at 210w(so plugs into any normal 15a outlet). BF16 KV, MTP enabled(no spec). ### Latency Metric Mean P50 P90 P95 P99 TTFT 4.605s 4.674s 4.682s 4.683s 4.690s TPOT…

Read the full story at r/LocalLLM ↗

Timeline · 2 reports

  1. 2026-11-06 06:38 · r/LocalLLM
    RadianceMemory + RadianceSwap: KV cache paged to host RAM by sparse attention (byte-identical output), and in-process multi-model serving that moves VRAM between models at runtime — 2× Radeon AI PRO R9700
  2. 2026-11-06 02:08 · r/LocalLLM
    Qwen3.8 27B FP8 on 4× Radeon AI PRO R9700: 7K tok/s prefill, 814 tok/s C8 decode

More stories

  1. Election backlash against data centers rattles AI stocks and expansion plans — TechCrunch AI
  2. Anthropic strikes $3.4bn deal for gas-fired generators to power AI data centres — Financial Times AI
  3. OpenAI to keep expanding AI data centers despite election results favoring restrictions — DigiTimes Asia
  4. Tencent AI chief warns AI will soon master biology, raising new safety risks — SCMP China Tech
  5. Hydra-PSIE expands 'Coeziune OM–IA' DOI registry to 39 published works — DEV Community — AI
  6. Developer's 48-Hour Field Notes Detail Free-Tier LLM Batch Job Failures — DEV Community — AI
  7. Apple's John Ternus reportedly pushing for faster AI progress beyond Siri upgrades — Techmeme
  8. US AI Safety Center Staff Now Report to White House, Raising Independence Concerns — AP News

Get the daily brief of stories like this at 6:30 every morning →