AINewsnow

I’m not sure how it happened, but on my system—equipped with an RTX 4060 (8 GB VRAM) and 32 GB of RAM—the Qwen3.6 35B A3B model managed to generate 23 tokens per second (23 TPS) at its maximum context window.

I know it took a long time to generate the first token. There is q4_0 quantization for the KV cache.

Read the full story at r/LocalLLM ↗

Timeline · 1 report

  1. 2026-10-18 19:43 · r/LocalLLM
    I’m not sure how it happened, but on my system—equipped with an RTX 4060 (8 GB VRAM) and 32 GB of RAM—the Qwen3.6 35B A3B model managed to generate 23 tokens per second (23 TPS) at its maximum context window.

More stories

  1. OpenAI brings frontier models and Codex to Google Cloud — OpenAI News
  2. OpenAI acquires Orchestra Bio to fold drug-development agents into Rosalind — OpenAI News
  3. OpenAI agents disabled their own safety monitoring systems, with one lying about it — Financial Times AI
  4. Solving a 31-year-old grand challenge in mathematics – the KLS conjecture — Microsoft Research Blog
  5. How U.S. Hunger connects leaders and data using Amazon Quick — AWS Machine Learning Blog
  6. NVIDIA Kumo Relational Outperforms Supervised Machine Learning on Relational Data, No Training Required — Hugging Face Blog
  7. Space X and Tesla exploring merger, CEO Elon Musk says — Axios AI+
  8. Anthropic pitches AI safety bill that may block a model if 50+ people could be killed — TechCrunch AI

Get the daily brief of stories like this at 6:30 every morning →