AINewsnow

Models keeps unloading from GPU

I have a dual gpu setup which is 16gb 5080 and 12gb 3060, and i am trying to load Qwen3.8 27b Q4 K XL, i tried Q6, but it was not compatible with dual gpu setup, now qwen3.8 27b work but keeps unloading while running, without any specific reason, i tried LM studio as well as llama server nothing wo…

Read the full story at r/LocalLLM ↗

Timeline · 1 report

  1. 2026-10-13 10:03 · r/LocalLLM
    Models keeps unloading from GPU

More stories

  1. I released Harness Studio, an open-source alternative to LM Studio and Ollama. Looking for feedback and contributors. — r/LocalLLM
  2. train the best model you can on your mac in one hour — r/learnmachinelearning
  3. Running decision models: Picking quant, VRAM usage — r/LocalLLaMA
  4. Problems with Tesla V100 SXM2 — r/LocalLLaMA
  5. I clustered two old Android phones to run a 27B LLM locally. Bypassed RAM limits via C++ RPC split and Hadamard Ternary math (No overheating) — r/LocalLLM
  6. I got Gemma 4 26B MoE running at 60+ tok/s on my RTX 4060 Laptop by caching hot experts on the GPU — r/LocalLLM
  7. Ternary Bonsai 2 27B on a $250 Intel Arc B580: writes a working snake game at 108 t/s, 89 t/s at 115K context (open source) — r/LocalLLM
  8. llama.cpp optimizations for specific hardware and models — r/LocalLLaMA

Get the daily brief of stories like this at 6:30 every morning →