AINewsnow

2× Arc Pro B70 on Windows: Qwen3.8-27B Q8_0 at 51 t/s average over 1,030 real agent requests (15K–193K context, temp 1.0). Recipe, patches, and a oneAPI leak fix

Setup: 2× Arc Pro B70 32 GB, Windows 11, llama.cpp SYCL with tensor parallel across both cards, DFlash2 drafter (Q4_K_M), 200K context, temp 1.0. Observed in real use: (from server logs: two working sessions, ~1,030 requests, 1.68M generated tokens of agentic coding): Context Decode Prompt processi…

Read the full story at r/LocalLLM ↗

Timeline · 1 report

  1. 2026-10-24 04:03 · r/LocalLLM
    2× Arc Pro B70 on Windows: Qwen3.8-27B Q8_0 at 51 t/s average over 1,030 real agent requests (15K–193K context, temp 1.0). Recipe, patches, and a oneAPI leak fix

More stories

  1. Ran DeepSeek V4.1 Flash 1017 on a single RTX 5090 — r/LocalLLaMA
  2. NVIDIA's Dory 1.0 4B (looped reasoning model) on Apple Silicon with MLX: 101 tok/s at 1 loop, 69 at 2, on an M5 Max (4.2 GB 8-bit build) — r/LocalLLM
  3. llama: create new fused op for QSA mask by am17an · Pull Request #30640 · ggml-org/llama.cpp — r/LocalLLaMA
  4. RecursiveSI/flint (40B A2.5B) · Hugging Face — r/LocalLLaMA
  5. Locality of reference helped 94GB Qwen3.8-Flash-Next runs at 68 tok/s on one RTX 5090 — r/ArtificialInteligence
  6. Offline manga translation on Android: bubble detection + OCR + a local LLM on llama.cpp, ~15 s/page on a Galaxy S23 (open source) — r/LocalLLM
  7. The irony of jev/decision models... — r/LocalLLM
  8. Vibecoding your own inference engine is easy they said — r/LocalLLaMA

Get the daily brief of stories like this at 6:30 every morning →