AINewsnow

Happy with 20-30 Tokens Per Second

Just realized the "speed paradox" in LLMs. I'm seeing ~20-30 tokens per second on my local setup running a well-optimized 26-30B model, which is essentially the same snappiness you get from massive 500B+ models running on heavy-duty server hardware. It’s fascinating that at the end of the day, for…

Read the full story at r/LocalLLM ↗

Timeline · 1 report

  1. 2026-10-20 15:04 · r/LocalLLM
    Happy with 20-30 Tokens Per Second

More stories

  1. OpenAI acquires Orchestra Bio — OpenAI News
  2. DeepSeek V4.1 Flash 1017 [GGUF + Abliterated] — r/LocalLLaMA
  3. CoreWeave Completes Acquisition of Weights & Biases — CoreWeave Blog
  4. Build a semantic layer for agentic AI on AWS with AtScale and Amazon Bedrock AgentCore — AWS Machine Learning Blog
  5. Structured outputs for Anthropic Claude models on Amazon Bedrock — AWS Machine Learning Blog
  6. DHS Orders DeepSeek Technology Removed From Federal Civilian IT Systems — Wall Street Journal Technology
  7. Ex-OpenAI worker accused of bugging offices planned to 'flee' to Saudi Arabia, prosecutors say — Sky News Technology
  8. FairStress: A deeper evaluation of fairness in LLM decision-making — Allen Institute for AI (Ai2)

Get the daily brief of stories like this at 6:30 every morning →