AINewsnow

Round 3: Tested 128K context on a 3090 with real code (Django) and a novel. Finding one thing is solid, counting is hopeless

People asked on my last post whether these models can actually use 128K or just fit it. So I tested it, on Moby-Dick and on the Django source since that's what most of us actually paste in. Setup: Qwen3.8 27B Q4_K_M, Qwen3.6 35B-A3B Q4_K_M, Nemotron 3.5 Bolt (2-bit), one 3090, llama.cpp, everything…

Read the full story at r/LocalLLM ↗

Timeline · 1 report

  1. 2026-10-18 14:59 · r/LocalLLM
    Round 3: Tested 128K context on a 3090 with real code (Django) and a novel. Finding one thing is solid, counting is hopeless

More stories

  1. NVIDIA Nemotron 3.5 Super VL Powers Specialized Agents with Multimodal Reasoning — NVIDIA Technical Blog
  2. Notes from a 2 AM On-Call: When llama-swap's Model Reload Hit a 30s Timeout Wall — DEV Community — Machine Learning
  3. Qwen Ships Qwen3-VL 8B Locally, Seeing Images and Video in 5 GB — AlphaSignal
  4. Efforts to upstream adaptive MTP depth to llama.cpp abandoned — r/LocalLLM
  5. New quant types suggested: IQ2_K, IQ3_K, IQ4_K, IQ5_K and IQ6_K — r/LocalLLaMA
  6. Has anyone used EXO in x86 machines, or am I just better off using llama.cpp or dllama? — r/LocalLLM
  7. Developer Mert Cobanov Runs llama.cpp on a Jailbroken PS5 — AlphaSignal
  8. Qwen3.8-27B at 42 tok/s single-user on one DGX Spark, fastest single-Spark entry on spark-arena, with 99 to 101% of BF16 quality — r/LocalLLM

Get the daily brief of stories like this at 6:30 every morning →