Two DGX Spark-class boxes, one model: a copy on each beat splitting it
For seven weeks our local language model ran split across two NVIDIA GB10 machines with tensor parallelism. On 22 October we replaced that with a full copy of the model on each machine and a least-connection proxy in front. Throughput went from about 180 to 210 tokens a second, and either machine c…
Read the full story at DEV Community — AI ↗
Timeline · 1 report
- 2026-11-12 14:00 · DEV Community — AI
Two DGX Spark-class boxes, one model: a copy on each beat splitting it