AINewsnow

Running Qwen3.8-27B at 128K Context on an RTX 5060 Ti 16GB + GTX 1080 8GB — 21.9 tok/s

Hi, I've managed to run ISTA DAS LAB Qwen3.8-27B IQ3_S with a 128K context window across an RTX 5060 Ti 16GB and a GTX 1080 8GB using a custom CUDA build of llama.cpp. The GTX 1080 (compute capability 6.1) wasn't supported by the stock backend I initially tried, so I built llama.cpp with support fo…

Read the full story at r/LocalLLM ↗

Timeline · 1 report

  1. 2026-10-14 13:11 · r/LocalLLM
    Running Qwen3.8-27B at 128K Context on an RTX 5060 Ti 16GB + GTX 1080 8GB — 21.9 tok/s

More stories

  1. common : add --spec-draft-n-ubatch by ggerganov · Pull Request #30459 · ggml-org/llama.cpp — r/LocalLLaMA
  2. A couple of hours of work - serious speed improvements — r/LocalLLM
  3. I clustered two old Android phones to run a 27B LLM locally. Bypassed RAM limits via C++ RPC split and Hadamard Ternary math (No overheating) — r/LocalLLM
  4. Research: Gemma 4 E2B on a Pixel 8 Pro (Tensor G3) with llama.cpp — r/LocalLLM
  5. Qwen 27B at ~20 tok/s on a jailbroken PS5's GPU. I forked PS5LM and turned it into a native app — r/LocalLLM
  6. Why AnythingLLM is moving from Ollama to Lemonade and llama.cpp — r/LocalLLaMA
  7. Quartermaster 1.7 update now lets you run llama.cpp's new decision models (/v1/systemone) with a playground tab — r/LocalLLaMA
  8. An 80B MoE (Qwen3-Next, 48 GB GGUF) answering in a Chrome tab: llama.cpp's WebGPU backend, experts paged in from disk — r/LocalLLaMA

Get the daily brief of stories like this at 6:30 every morning →