AINewsnow

boro64/Ternary-Nemotron-3.5-Lightning-30B-A3B-GGUF · Hugging Face

Ternary Nemotron 30B-A3B — 368K Context on 12 GB GPU (GGUF) A ternary (1.58-bit) 30B-A3B MoE model that runs on a 12 GB GPU with 368K context, 172 tok/s decode and 7000+ tok/s prefill — in both Stock llama.cpp and the PrismML fork. One file, both engines. Engines ✅ Stock llama.cpp (validated on b11…

Read the full story at r/LocalLLaMA ↗

Timeline · 2 reports

  1. 2026-10-26 08:51 · r/LocalLLaMA
    tencent/Hy-SimulMT-30B-A3B · Hugging Face
  2. 2026-10-26 07:49 · r/LocalLLaMA
    boro64/Ternary-Nemotron-3.5-Lightning-30B-A3B-GGUF · Hugging Face

More stories

  1. Full DeepSeek-V4.1-Flash-1017 (510 GB, original weights) on a 64 GB RAM + RTX 3060 12 GB PC: up to 7.4 tok/s, 98.3% agreement with the full model. Engine released (Apache-2.0) — r/LocalLLM
  2. Qwen 3.8 Flash Next Gyro Coder is out! — r/LocalLLaMA
  3. qwen4exp : build the QSA selection without the kq_mask by sanmai · Pull Request #30965 · ggml-org/llama.cpp — r/LocalLLaMA
  4. Backup your quants! — r/LocalLLaMA
  5. Qwen3.8-27B Q6 on one RTX 5090: 69 → 100 tok/s short, 35 → 64 tok/s at 120K context, just by leaving LM Studio for a native llama.cpp build (full benchmarks + recipe) — r/LocalLLM
  6. I created a benchmark to test local LLMs on your Snapdragon NPU (open source) — r/LocalLLM
  7. I built my first RAG project using Whisper, BGE-M3 and Ollama — what should I improve? — r/learnmachinelearning
  8. Gemma 4 26B-A4B on a 16 GB Mac: 33/40 on TypeScript tasks, 31 tok/s, offline, using Google's stock file — r/huggingface

Get the daily brief of stories like this at 6:30 every morning →