AINewsnow

769 tok/s on a Single RTX 4090: Spark-X2.6-1.7B DSpark

769 tokens per second on a single NVIDIA RTX 4090. Spark-X2.6-1.7B with DSpark reaches that peak in the published GSM8K benchmark using a Q4_K_M target with llama.cpp, up from 424 tok/s without DSpark. For BF16, the highest reported speedup is 2.43x , also on GSM8K and RTX 4090: 220 → 536 tok/s . T…

Read the full story at DEV Community — Machine Learning ↗

Timeline · 1 report

  1. 2026-11-01 13:39 · DEV Community — Machine Learning
    769 tok/s on a Single RTX 4090: Spark-X2.6-1.7B DSpark

More stories

  1. NVIDIA RTX Spark: Grace CPU and Blackwell GPU on one package, up to 128GB shared memory — r/ArtificialInteligence
  2. Spark-X2.6-4B DSpark Is Here: More Than 2x Average BF16 Inference Speed — DEV Community — Machine Learning
  3. Nvidia's $3 billion Meta investment adds to web of AI bets — Barron's
  4. Halving the KV Cache Doubles Capacity and Buys 1% of Speed — DEV Community — Machine Learning
  5. How we fit a 27B model on a 12 GB laptop GPU — DEV Community — Machine Learning
  6. Run a 35B to 180B LLM Without a GPU: VIDRAFT POCKET Passes 3.15 Million Downloads — DEV Community — Machine Learning
  7. How to Run LFM3-Omni-2B GGUF Locally (Only llama.cpp Makes It Talk Back) — DEV Community — Machine Learning
  8. A game scripting language that runs speech-to-text 5x faster than llama.cpp on CPU — Lobsters AI

Get the daily brief of stories like this at 6:30 every morning →