AINewsnow

My foray into local ai. Two BC-250 ex mining apus running Qwen3.6-35B-A3B Q4_K_M at 60 tok/s with 64k context

These boards cost me $115 each and I have them connected using llama.cpp with Vulkan and RPC on Bazzite. The boards have roughly 27GB of combined GPU memory and communicate over 1gb Ethernet. For around $300 including psu I’m loving the performance. I have a few more and want to see what 6 looks li…

Read the full story at r/LocalLLaMA ↗

Timeline · 1 report

  1. 2026-09-24 06:25 · r/LocalLLaMA
    My foray into local ai. Two BC-250 ex mining apus running Qwen3.6-35B-A3B Q4_K_M at 60 tok/s with 64k context

More stories

  1. Transformers now runs llama.cpp quants — Hugging Face Blog
  2. yandex/AliceAI-Foundation-80B-A3B-Base: Russian-developed competitor to Qwen 35B and DeepSeek V4 Flash — r/LocalLLaMA
  3. Performance tune for gemma4-26b-a4b flash attention shape. by frobnitzem · Pull Request #28450 · ggml-org/llama.cpp · GitHub — r/LocalLLaMA
  4. I trained a 360M-param Python model from scratch on two workstation GPUs and wrote up every step, including the bugs — r/learnmachinelearning
  5. Stanford's MAttr Tops AI Interpretability Benchmark by Nearly 3x — AlphaSignal
  6. I turned Qwen3.8-27B Q2_64 + llama.cpp into a fully TypeSafe AI-compatible Jev-like system. OpenAI API still intact! World’s first Vision-enabled Jev-like model! <10 GB VRAM, 170 ms on an RTX 3090 and ~140 tok/s in chat. 76% vs. 88% Jev-1.13 Acc. on a diverse 22,000-request typed-decision benchmark — r/LocalLLaMA
  7. GGUFs in transformers natively! — r/LocalLLaMA
  8. 2× Tesla P100 (2016 cards) in 2026: 110 tok/s on a 30B MoE, 16 tok/s at 1M context — r/LocalLLM

Get the daily brief of stories like this at 6:30 every morning →