Self-hosting a 35B MoE for a 20-person team on one desktop box: what it took and what it does (all open source)
We've been running our own LLM for the team for about a week, and I wanted to share how it turned out, since "can one machine actually serve a whole team?" comes up here a lot. Short answer: yes, if you pick the right model. Hardware and model One NVIDIA DGX Spark (128GB unified memory) Ornith-1.5-…
Read the full story at r/LocalLLM ↗
Timeline · 1 report
- 2026-10-06 02:59 · r/LocalLLM
Self-hosting a 35B MoE for a 20-person team on one desktop box: what it took and what it does (all open source)