AINewsnow

Geezuz kernel: Qwen3.8-27B at 190 tok/s (620~ tok/s at 8 requests) on two 3090s

Geezuz, its fast! Qwen3.8-27B, two RTX 3090s power-limited to 280 W, plain PCIe, no NVLink, at 0 Context: (tests showing results with context are found on the next table) Requests Q4_K_M Q8_0 1 request 190 tok/s 143 tok/s 8 requests at once (total) 622 tok/s 610 tok/s llama.cpp's fastest two-GPU se…

Read the full story at r/LocalLLaMA ↗

Timeline · 1 report

  1. 2026-10-20 13:46 · r/LocalLLaMA
    Geezuz kernel: Qwen3.8-27B at 190 tok/s (620~ tok/s at 8 requests) on two 3090s

More stories

  1. DeepSeek releases DeepSeek-V4.1-Flash-1017 open weights under the MIT license — r/huggingface
  2. 32GB to 64GB of RAM took Qwen3.8-Flash-Next from IQ2_XS (2-bit) to Unsloth UD-IQ4_XS (4-bit) on a single 4090 — r/LocalLLaMA
  3. 12GB VRAM + 16GB RAM is enough for very smart local AI with 120K+ context — r/LocalLLM
  4. Qwen3-4B in 1.24 GB at perplexity 16.6 (IQ2_XXS at the same size: 28.6) - a tiny 8-state trellis quant for llama.cpp — r/LocalLLM
  5. llama.cpp support for Cohere models — r/LocalLLaMA
  6. Budget rig for local LLM inference + distillation: 2x RTX 3060 12GB vs 2x Tesla P100 16GB? — r/LocalLLM
  7. Is there an alternative to base llama.cpp for RDNA2 gpus in general with local LLM usage ? — r/LocalLLaMA
  8. llama server's default port changed from 8080 to 9931 — r/LocalLLaMA

Get the daily brief of stories like this at 6:30 every morning →