AINewsnow

Used Opus 5.5 to optimize llama.cpp inference for Swift Qwen 3.8 27B Q6_K on RTX 5090 - decode 143 tok/s prefill 2840 tok/s

This story is from 2026-10-01. It is preserved in the archive; the latest stories are on the live feed.

https://preview.redd.it/uflbt6x61tsh1.png?width=472&format=png&auto=webp&s=20499138331923a3d84505e7fa530d02d1a37978 https://preview.redd.it/toxiq4ah1tsh1.png?width=419&format=png&auto=webp&s=5cdcc3244accdeee500ddac8a5cac43e63c54851 https://preview.redd.it/21lkvj782tsh1.png?width=439&format=png&auto…

Read the full story at r/LocalLLM ↗

Timeline · 1 report

  1. 2026-10-01 07:09 · r/LocalLLM
    Used Opus 5.5 to optimize llama.cpp inference for Swift Qwen 3.8 27B Q6_K on RTX 5090 - decode 143 tok/s prefill 2840 tok/s

More stories

  1. Running Llama 3.2 & Qwen 2.5 100% Offline in React Native & Expo 52: Solving Thermal Throttling, OOMs, and Background Streaming — DEV Community — AI
  2. DeepSeek API Expansions, Local 284B MoE Runs, and New Developer Tooling — DEV Community — Machine Learning
  3. Study audits major LLMs for racial and sex bias in pain management — arXiv cs.CL
  4. Off Grid AI Apps Bring Local LLMs and Image Generation to Phones and Macs — DEV Community — AI
  5. AI Daily Digest · November 14, 2026: NVIDIA Loses Its Founder-CEO, ChatGPT Gets a Wallet, llama.cpp Hits 1.0 — DEV Community — AI
  6. Kindi – fast local inference on low-cost GPUs, and the exact-match rule that makes its numbers trustworthy — DEV Community — Machine Learning
  7. Show HN: 53ON AI Server – OpenAI-compatible server for AMD Ryzen AI Max — DEV Community — AI
  8. Weight-space programming is not autonomous self-improvement — DEV Community — Machine Learning

Get the daily brief of stories like this at 6:30 every morning →