AINewsnow

Just update of ik_llama.cpp fork and seek of feedback

Continuation of https://www.reddit.com/r/LocalLLaMA/comments/1wynomx/k_llamacpp_moe_optimizations_expert_residency/ This time I finally get speeds compared to Strata for ik_llama.cpp on 6gb vram rtx 2060 ( generates about 45–48 t/s on short prompts and about 37 t/s at 28K context. Prefill runs at a…

Read the full story at r/LocalLLaMA ↗

Timeline · 1 report

  1. 2026-10-24 16:34 · r/LocalLLaMA
    Just update of ik_llama.cpp fork and seek of feedback

More stories

  1. Kimi K3.1 Pro and Kimi K3.1 Pro Highspeed are available now in AI Gateway — Vercel Blog
  2. Securing our frontier model training environment — OpenAI News
  3. DeepSeek V4 Flash API prices compared across providers (Oct 2026) — r/huggingface
  4. Ai in a nutshell — r/LocalLLM
  5. Block unveils Dessa, a "true speech-to-speech" AI model trained on "nothing but human conversation" that it says runs locally on a MacBook Pro M4 Max or newer — Techmeme
  6. UK Vows £100 Million for Biosecurity Lab With OpenAI, Anthropic — Bloomberg AI
  7. Fired OpenAI researchers say independent safety audits aren't going well — NPR Technology
  8. Qwen 3.8 Flash Next GSQ-RCO IQ_3S at 70tok/s decode/1900tok/s prefill on 2013 Xeon E5-1680v2, 128GB quad channel DDR3-1866 ECC, and an EVGA 3090 FTW3 — r/LocalLLM

Get the daily brief of stories like this at 6:30 every morning →