AINewsnow

Qwen3-Coder 30B on RTX 3080 20GB — KV cache stability and context tuning

I’m running Qwen3-Coder-30B-A3B-Instruct Q3_K_M GGUF in llama.cpp on an RTX 3080 20GB for OpenCode. My stable config so far is: K cache: q8_0 V cache: f16 Flash Attention: off GPU layers: all Parallel: 1 Threads: 8 Continuous batching: on I originally tried: K: q8_0 V: q8_0 Flash Attention: on but…

Read the full story at r/LocalLLM ↗

Timeline · 1 report

  1. 2026-09-24 23:05 · r/LocalLLM
    Qwen3-Coder 30B on RTX 3080 20GB — KV cache stability and context tuning

More stories

  1. Performance tune for gemma4-26b-a4b flash attention shape. by frobnitzem · Pull Request #28450 · ggml-org/llama.cpp · GitHub — r/LocalLLaMA
  2. Qwen3.8-Flash-Next on 12GB VRAM - 65 tokens per second — r/LocalLLaMA
  3. Self-Hosted LLM: Essential Llama TCO Comparison Guide — DEV Community — AI
  4. PSA: llama.cpp -cram should be increased for agentic workflows (default is 8192) — r/LocalLLaMA
  5. Gufo: the all-in-one strix halo inference engine — r/LocalLLM
  6. My foray into local ai. Two BC-250 ex mining apus running Qwen3.6-35B-A3B Q4_K_M at 60 tok/s with 64k context — r/LocalLLaMA
  7. Stanford's MAttr Tops AI Interpretability Benchmark by Nearly 3x — AlphaSignal
  8. I turned Qwen3.8-27B Q2_64 + llama.cpp into a fully TypeSafe AI-compatible Jev-like system. OpenAI API still intact! World’s first Vision-enabled Jev-like model! <10 GB VRAM, 170 ms on an RTX 3090 and ~140 tok/s in chat. 76% vs. 88% Jev-1.13 Acc. on a diverse 22,000-request typed-decision benchmark — r/LocalLLaMA

Get the daily brief of stories like this at 6:30 every morning →