AINewsnow

I made my local LLM start thinking while I'm still typing: 12s → 3s wait on a 6 GB GPU (open source)

Named after Mafdet, the ancient Egyptian feline goddess known as "the runner", who struck snakes and scorpions before they could attack. Reasoning models are great until you're on a mid-range GPU: you hit Enter, then wait 10+ seconds while the model thinks before the first word. Meanwhile, while yo…

Read the full story at r/LocalLLM ↗

Timeline · 1 report

  1. 2026-10-22 15:18 · r/LocalLLM
    I made my local LLM start thinking while I'm still typing: 12s → 3s wait on a 6 GB GPU (open source)

More stories

  1. Uber brings ChatGPT Work to all employees with OpenAI — OpenAI News
  2. OlmoEarth v1.3: State-of-the-art embeddings for the whole planet — Allen Institute for AI (Ai2)
  3. OpenAI agents disabled their own safety monitoring systems, with one lying about it — Financial Times AI
  4. Anthropic Joins the A.I. Agent Race With Claude Duet — New York Times Technology
  5. Anthropic donates the Agent Skills open standard to the Agentic AI Foundation and says 70+ platforms have integrated it, after donating MCP to the AAIF in 2025 (Anthropic) — Techmeme
  6. Amazon Polly delivers a major upgrade to its Generative engine — AWS Machine Learning Blog
  7. Write your own training loop — without managing GPUs — with Amazon SageMaker training session APIs — AWS Machine Learning Blog
  8. Canva releases its first image generation model — TechCrunch AI

Get the daily brief of stories like this at 6:30 every morning →