AINewsnow

4 coding agents on one RTX 3090: ~290 tok/s total from a custom CUDA engine for Qwen3.8-27B (3-4x llama-server --parallel)

I've been building OpenJet, a local coding agent with its own CUDA megakernel engine for Qwen3.8-27B Q4_K_M. Until now it served one conversation at a time. The new release serves several agents at once on a single GPU. https://reddit.com/link/1xgs8hp/video/p0zd2rxse0yh1/player Numbers (RTX 3090, Q…

Read the full story at r/LocalLLaMA ↗

Timeline · 1 report

  1. 2026-10-27 13:15 · r/LocalLLaMA
    4 coding agents on one RTX 3090: ~290 tok/s total from a custom CUDA engine for Qwen3.8-27B (3-4x llama-server --parallel)

More stories

  1. Agentes LLM: una cola de evidencia para email — DEV Community — AI
  2. Local Qwen 3.8 27B as the worker, Opus as the planner: same 38 tests passed, half the API bill — r/LocalLLM
  3. Why does my local LLM keep reverting to generic prose despite extensive author-style examples and prompt engineering? Am I overengineering this? — r/LocalLLM
  4. Janas: a C inference engine that runs MoE models bigger than your RAM by streaming experts from NVMe, plus an almost complete OpenAI-compatible server, MCP tools, vision and speech (GPL-3.0) — r/LocalLLM
  5. I tested a 12B open-source AI Humanizer locally. Here's what I found. — r/learnmachinelearning
  6. I tested LFM2.5-24B-A2B vs Qwen3.6-35B-A3B on 75 fresh multi-step tool tasks. Qwen solved 91%, LFM2.5 49%, and LFM2.5 said "done" on most of its failures — r/LocalLLM
  7. I trained LLaMa from scratch — r/deeplearning
  8. Basalt (NVFP4) vs Unsloth Studio (llama.cpp IQ4_XS) — 5090 with 64GB RAM — r/LocalLLM

Get the daily brief of stories like this at 6:30 every morning →