AINewsnow

Benchmarked Strata 0.1.25 and 0.1.39 against llama.cpp on three cards: an RTX PRO 6000 (96 GB), a 3090 Ti and a 5090 laptop

TL;DR: on Qwen3.8-Flash-Next (IQ3_S), a model too big for a 24 GB card, Strata gave the first word about ten times sooner than llama.cpp on a 5090 laptop (1.9 against 19.1 s at a 4K prompt) and 8.8 times sooner on a 3090 Ti (1.92 against 16.87 s), and wrote 3.5 and 4.4 times as fast. On an RTX PRO…

Read the full story at r/LocalLLM ↗

Timeline · 1 report

  1. 2026-10-12 16:43 · r/LocalLLM
    Benchmarked Strata 0.1.25 and 0.1.39 against llama.cpp on three cards: an RTX PRO 6000 (96 GB), a 3090 Ti and a 5090 laptop

More stories

  1. I clustered two old Android phones to run a 27B LLM locally. Bypassed RAM limits via C++ RPC split and Hadamard Ternary math (No overheating) — r/LocalLLM
  2. Qwen3.8-Flash-Next (125B) at ~100 tok/s on an M5 Ultra Mac Studio with llama.cpp — r/LocalLLM
  3. Qwen 3.8 27B Q4_K_M running on a OnePlus 13 with 24 GB RAM — r/LocalLLM
  4. BrokenML: can agents fix broken ML jobs? — r/LocalLLaMA
  5. strix-llama new branch: prompt processing +21–54% over Vulkan and up to 2.7× over stock ROCm kernel at 128k, same outputs quality — r/LocalLLaMA
  6. Do you actually test open-model inference compatibility? — r/LocalLLaMA
  7. Qwen3.8-27B on a single 3090: 140 tok/s on code with a custom megakernel — r/LocalLLaMA
  8. PSA: DeepSeek V4.1 Flash habitually exfiltrates API keys. It is dangerously misaligned and may be hazardous to use — r/LocalLLaMA

Get the daily brief of stories like this at 6:30 every morning →