AINewsnow

Bonsai 2 27B at 80tps+ on a 5060 ti 16gb

https://reddit.com/link/1x0n63s/video/as3g9py8w7uh1/player ChatGPT and Claude worked to modify and optimise ninfer from here to run on a 5060 ti with Bonsai 2 27B. Prefill is over 1000tps with decode above 50ish tps for most of a 262k context window.

Read the full story at r/LocalLLM ↗

Timeline · 1 report

  1. 2026-10-08 10:13 · r/LocalLLM
    Bonsai 2 27B at 80tps+ on a 5060 ti 16gb

More stories

  1. OpenAI will watermark ChatGPT outputs by default—but only in the EU — Ars Technica AI
  2. Claude Pro vs ChatGPT Plus vs Copilot Premium: which one would you choose for this use case? — r/ChatGPTPro
  3. What's up, Docsy? Google’s docs project joins the Linux Foundation as AI agents become readers — The New Stack AI
  4. Finished a CPD-accredited AI mastery program — here are the 5 things actually worth knowing — r/ChatGPT
  5. Context windows like Claude Code? — r/ChatGPTPro
  6. 😺 ChatGPT's GPT-6 now builds calculators — The Neuron
  7. Gemini app needs to improve — r/GeminiAI
  8. Why is Gemini so drastically different to Claude or ChatGPT? — r/GeminiAI

Get the daily brief of stories like this at 6:30 every morning →