AINewsnow

Strata looping badly with iq2_xxs

Hello, I'm running strata with iq2_xxs quant and indeed I am quite amazed to be able to run it at speeds comparable to vanilla llama.cpp + 3.8 27B. I'm getting about 30 tps on both. Strata seems to offer higher peak tps when working with code, up to 50. Although I do feel that the bigger model offe…

Read the full story at r/LocalLLM ↗

Timeline · 1 report

  1. 2026-10-04 15:30 · r/LocalLLM
    Strata looping badly with iq2_xxs

More stories

  1. RTX 5070 TI and 32GB RAM DDR4, WHAT IS THE BEST VIDEO MODEL I COULD USE? — r/comfyui
  2. llama, server: add /v1/systemone API (models: laya, julia-1, lev, openjev, kev) by ngxson · Pull Request #29818 · ggml-org/llama.cpp — r/LocalLLaMA
  3. Running 95.5 GiB Qwen3.8-Flash-Next at 41–52 tok/s on a 64GB Mac (1.76x faster than llama.cpp): Slipstream release, 130k context scaling, + Swift variant — r/LocalLLaMA
  4. I want to hook llama.cpp up to a search provider, both locally hosted. I am hitting nothing but dead-ends. Help? — r/LocalLLM
  5. From 1x3090 to 20 DGX Sparks: my house fuses were the first bottleneck — r/LocalLLaMA
  6. Need maybe say "Use llama.cpp" — r/LocalLLaMA
  7. I got llama.cpp inference running on the Snapdragon 8 Gen 3 Hexagon NPU from non-root Termux + Adreno OpenCL results (S24 Ultra) — r/LocalLLM
  8. Strata takes the promise of "MoE models just need a total amount of VRAM+RAM" and makes it a reality — r/LocalLLM

Get the daily brief of stories like this at 6:30 every morning →