AINewsnow

shura.cpp: a llama.cpp fork with a VRAM governor, so a 35B MoE can share one 16 GB card with the desktop you keep using

I've been running Tiel-Coder-35B-A3B on my RX 7800 XT. The model itself runs fine, but using the same PC for anything else was getting annoying. If the model filled most of the VRAM, opening a video in Brave or joining a call could make the whole desktop stutter. Sometimes the compositor would get…

Read the full story at r/LocalLLM ↗

Timeline · 1 report

  1. 2026-10-28 09:34 · r/LocalLLM
    shura.cpp: a llama.cpp fork with a VRAM governor, so a 35B MoE can share one 16 GB card with the desktop you keep using

More stories

  1. I tested LFM2.5-24B-A2B vs Qwen3.6-35B-A3B on 75 fresh multi-step tool tasks. Qwen solved 91%, LFM2.5 49%, and LFM2.5 said "done" on most of its failures — r/LocalLLM
  2. LiquidAI/LFM3-Omni-2B-GGUF — r/LocalLLaMA
  3. Qwen3.8-27B Q6 on one RTX 5090: 69 → 100 tok/s short, 35 → 64 tok/s at 120K context, just by leaving LM Studio for a native llama.cpp build (full benchmarks + recipe) — r/LocalLLM
  4. llmserver – one bash script for local LLMs with model security checks (llama.cpp / vLLM / Ollama) — r/huggingface
  5. Introducing Zest: model server + agent + friendly GUI — r/LocalLLaMA
  6. I made Pumpkin, a graphical manager for llama.cpp for my girlfriend. It turned out well, so I put it on github — r/LocalLLaMA
  7. Nanbeige4.3-33B Q4 on a MacBook Pro M5 Max, mlx-lm vs llama.cpp vs Kapitsa, Prefill/Decode 512 to 32K range. — r/LocalLLaMA
  8. Quad 3090 vs 256 M5 Ultra For Agentic Coding — r/LocalLLM

Get the daily brief of stories like this at 6:30 every morning →