AINewsnow

Now that some time has passed: which Qwen3.8 27B quant do you stick with?

Until recently I had a single 16GB GPU that I ran the IQ3_S version of Swift-1.5-Qwen3.8-27B-GSQ-RCO-GGUF on with strict context limitations. But now I got my hands on two RX7900XTs, so I have a pooled 40GB of VRAM available. I‘ve used llama.cpp so far. So even though Qwen4 is around the corner, I…

Read the full story at r/LocalLLM ↗

Timeline · 1 report

  1. 2026-11-04 14:45 · r/LocalLLM
    Now that some time has passed: which Qwen3.8 27B quant do you stick with?

More stories

  1. I built a one-click installer for local LLMs on Windows: it picks a model for your hardware and sets up Ollama — DEV Community — AI
  2. Mistral Large 4 Is Open Weight. Read the License and Do the Memory Math Before You Self-Host — DEV Community — AI
  3. Configuring llama.cpp for Enterprise Environments: A Comprehensive Guide to Concurrency and Stability — DEV Community — AI
  4. Independent study tests Direct Nash Optimization for aligning language models — DEV Community — Machine Learning
  5. The 40% Performance Tax of Ollama: Why Developers Are Moving Back to Raw llama.cpp and vLLM — DEV Community — AI
  6. I track how much each provider charges for the same open LLM: Llama 3.3 70B ranges from $0.155 to $0.78 per 1M tokens — DEV Community — AI
  7. [llama.cpp fork]: QFN + [specific quant] + [ultra specific hardware config] = [incredible and irreproducible pp and tg] — r/LocalLLaMA
  8. Anthropic open-sources FlashProve, zkSNARK proofs for 400B Llama 4 Maverick inference (110 tok/s on 1 H100) — r/machinelearningnews

Get the daily brief of stories like this at 6:30 every morning →