AINewsnow

DeepSeek 1017 prefill: 1147.2 → 1974.7 tok/s, decode: 32 → 73.7 tok/s. Here's my recipe.

I heard that DeepSeek's new Flash model was really good, so I downloaded the IQ1_M version to my local server right away. I wanted to deploy it on a single L40S with 48GB of VRAM. Naturally, I went with Strata v0.2.6, but I wasn't happy with the speed. I asked 6.1 Sol to act as a professional model…

Read the full story at r/LocalLLM ↗

Timeline · 1 report

  1. 2026-10-26 03:09 · r/LocalLLM
    DeepSeek 1017 prefill: 1147.2 → 1974.7 tok/s, decode: 32 → 73.7 tok/s. Here's my recipe.

More stories

  1. Moonshot AI launches Kimi K3.1 Pro with better coding skills, reduced cyber risks — Mint AI
  2. Maya-XS runs DeepSeek-V4.1-Flash in 120.6 GB, with all 384 experts resident in GPU memory on a 128 GB Ryzen AI Max. — r/LocalLLaMA
  3. Weapon or armour? The contentious debate over China’s open-weight AI — South China Morning Post Tech
  4. HyScale: Scaling Long-Context Training for Learned Sparse Attention — arXiv cs.AI
  5. qwen4exp : build the QSA selection without the kq_mask by sanmai · Pull Request #30965 · ggml-org/llama.cpp — r/LocalLLaMA
  6. Anyone have experience with deterring heat thieves? — r/LocalLLM
  7. I need to fine tune Qwen3.5-9B with QLora on about 500 GB of raw data (images/scanned documents/pdfs/word/excel) - what is the best way forward? — r/LocalLLaMA
  8. Monolithic ddr3 server — r/LocalLLaMA

Get the daily brief of stories like this at 6:30 every morning →