AINewsnow

Every 1,000 tokens of context costs 0.39 ms: decode speed from 1K to 100K on RTX Spark

This story is from 2026-10-29. It is preserved in the archive; the latest stories are on the live feed.

On my Surface Laptop Ultra, qwen3-coder:30b's time per token grows in a straight line with the context it has to read: 11.8 ms plus 0.39 ms for every 1,000 tokens of f16 cache. That's within 7% of what memory bandwidth predicts, so the bytes-per-token model from my first RTX Spark post holds for th…

Read the full story at DEV Community — AI ↗

Timeline · 1 report

  1. 2026-10-29 15:22 · DEV Community — AI
    Every 1,000 tokens of context costs 0.39 ms: decode speed from 1K to 100K on RTX Spark

More stories

  1. 8 examples of Gemini 4 Argon in action — Google Gemini Blog
  2. Welcome Le Chonk: Mistral Large 4 on Hugging Face — Hugging Face Blog
  3. Voice AI in vogue: Nvidia releases new open transcription model — TechCrunch AI
  4. Lend and borrow resources across teams with AWS Batch’s quota management for SageMaker Training jobs — AWS Machine Learning Blog
  5. How MoEngage turned over 400 APIs into agent tools with AgentCore Gateway — AWS Machine Learning Blog
  6. You can now generate, preview, and edit documents, PDFs, presentations, spreadsheets, and more directly in Gemini. — Google Gemini Blog
  7. Accelerate agentic AI with NVIDIA Nemotron models on Amazon EKS and SageMaker HyperPod EKS — AWS Machine Learning Blog
  8. Into the Omniverse: How Developers Turn Ideas Into Simulations With Frontier AI Agents — NVIDIA Blog

Get the daily brief of stories like this at 6:30 every morning →