Hot experts, cold RAM: running a big MoE on a machine with more memory than VRAM
Two budgets, and only one of them is expensive Every local deployment spends against two budgets at once: the size of the model and the price of the memory holding it. VRAM is the one people reach for first, and it is the one that costs the most per gigabyte. System DDR is fast, vast and cheap; GPU…
Read the full story at DEV Community — AI ↗
Timeline · 2 reports
- 2026-11-18 07:12 · DEV Community — AI
Hot experts, cold RAM: running a big MoE - 2026-11-17 00:22 · DEV Community — AI
Hot experts, cold RAM: running a big MoE on a machine with more memory than VRAM