AINewsnow

RTX PRO 4000 Blackwell (24GB) - nvfp4 vs GGUF for long context (100k+)?

Hello everyone, I’m currently running an RTX PRO 4000 Blackwell SFF (24GB VRAM). I'm trying to optimize for a minimum context window of ~100k tokens using a Qwen 3.8-27B model. Current Setup: Running vLLM with the GGUF plugin using Qwen3.8-27B-UD-Q4_K_XL.gguf . This allows me to hit a context size…

Read the full story at r/LocalLLM ↗

Timeline · 1 report

  1. 2026-09-21 07:54 · r/LocalLLM
    RTX PRO 4000 Blackwell (24GB) - nvfp4 vs GGUF for long context (100k+)?

More stories

  1. Qwen Image 2.1 PR to ComfyUI — r/StableDiffusion
  2. Alibaba’s Qwen releases Qwen3.8-Omni-Flash with 1M-token context — TechNode
  3. Alibaba's open-weight Qwen-Image-2.1 claims to beat closed models in image generation with just 7 billion parameters — The Decoder
  4. Qwen q4 3.8 27b 16 tok/s 32k RTX 3060 :D — r/LocalLLM
  5. 10 hours left fo Qwen Image 2.1 Public Open Source Release — r/StableDiffusion
  6. Qwen 3.8 27B running on a single RTX 5090 researches and creates a full animation using only code. — r/artificial
  7. US government website used Chinese model the FBI called "malicious" — Ars Technica AI
  8. M2 Mac ultra128gb Qwen flash next — r/LocalLLM

Get the daily brief of stories like this at 6:30 every morning →