AINewsnow

[Project] vLLM on Windows ARM64 for NVIDIA RTX Spark (with benchmarks)

Hi everyone! I've been working on getting vLLM running on the NVIDIA RTX Spark (Windows ARM64). I finally got it working with Qwen3.8-Flash-Next NVFP4, including MTP speculative decoding and CUDA graphs. Some benchmark results: 70.44 tokens/s in SPEED-Bench coding (vs. 35.37 with NVIDIA's llama.cpp…

Read the full story at r/LocalLLM ↗

Timeline · 1 report

  1. 2026-10-31 10:14 · r/LocalLLM
    [Project] vLLM on Windows ARM64 for NVIDIA RTX Spark (with benchmarks)

More stories

  1. 769 tok/s on a Single RTX 4090: Spark-X2.6-1.7B DSpark — DEV Community — Machine Learning
  2. Spark-X2.6-4B DSpark Is Here: More Than 2x Average BF16 Inference Speed — DEV Community — Machine Learning
  3. LFM3-1.7b seems impressive — r/LocalLLaMA
  4. Halving the KV Cache Doubles Capacity and Buys 1% of Speed — DEV Community — Machine Learning
  5. Are bespoke inference engines overhyped? My Strata experience on 2× R9700 — r/LocalLLaMA
  6. How we fit a 27B model on a 12 GB laptop GPU — DEV Community — Machine Learning
  7. Qwen3.6-35B-A3B on a laptop RTX 5050 8GB: ~1000 tok/s prefill and ~40-46 tok/s decode (config + steps) — r/LocalLLM
  8. Run a 35B to 180B LLM Without a GPU: VIDRAFT POCKET Passes 3.15 Million Downloads — DEV Community — Machine Learning

Get the daily brief of stories like this at 6:30 every morning →