AINewsnow

Janas: a C inference engine that runs MoE models bigger than your RAM by streaming experts from NVMe, plus an almost complete OpenAI-compatible server, MCP tools, vision and speech (GPL-3.0)

Qwen3-Next-80B (48 GB) on a laptop with 32 GB of RAM: 26 tok/s writing, where llama.cpp manages 3.0. That is the core idea of Janas-LLM, the first component of Janas, a family of AI tools I'm writing in plain C, no dependencies beyond the C library. How it works. The weights of a mixture-of-experts…

Read the full story at r/LocalLLM ↗

Timeline · 1 report

  1. 2026-10-29 20:41 · r/LocalLLM
    Janas: a C inference engine that runs MoE models bigger than your RAM by streaming experts from NVMe, plus an almost complete OpenAI-compatible server, MCP tools, vision and speech (GPL-3.0)

More stories

  1. I tested LFM2.5-24B-A2B vs Qwen3.6-35B-A3B on 75 fresh multi-step tool tasks. Qwen solved 91%, LFM2.5 49%, and LFM2.5 said "done" on most of its failures — r/LocalLLM
  2. llmserver – one bash script for local LLMs with model security checks (llama.cpp / vLLM / Ollama) — r/huggingface
  3. 8 examples of Gemini 4 Argon in action — Google Gemini Blog
  4. Let a small model critique and revise its own answers, then fine-tune on the revisions (on a rented GPU) — DEV Community — Machine Learning
  5. Amazon AWS posts 39% revenue growth as cloud rivals also accelerate — CNBC Technology
  6. Local Qwen 3.8 27B as the worker, Opus as the planner: same 38 tests passed, half the API bill — r/LocalLLM
  7. Why does my local LLM keep reverting to generic prose despite extensive author-style examples and prompt engineering? Am I overengineering this? — r/LocalLLM
  8. I tested a 12B open-source AI Humanizer locally. Here's what I found. — r/learnmachinelearning

Get the daily brief of stories like this at 6:30 every morning →