AINewsnow

A 763B MoE on two 96 GB GPUs: put the idle experts in host RAM, then measure per stream

The model has 763B parameters, the server has two graphics cards, and the gap was about six gigabytes per card. DeepSeek-V4.1-Flash, even compressed to 3 bits, wanted 102.1 GiB of VRAM (the memory on the GPU itself) on each of two RTX PRO 6000 Blackwell cards, and each card offered 95.6 GiB. Two la…

Read the full story at DEV Community — Machine Learning ↗

Timeline · 1 report

  1. 2026-10-16 05:33 · DEV Community — Machine Learning
    A 763B MoE on two 96 GB GPUs: put the idle experts in host RAM, then measure per stream

More stories

  1. Microsoft unveils MAI-Base-2 MoE model with 2.8T parameters — Techmeme
  2. Community tests Qwen 3.8 Flash Next on consumer hardware — r/LocalLLM
  3. PLEASE HELP !! Sapphire AMD Radeon AI Pro R9700 32GB Graphic Card — r/comfyui
  4. Small experiment: filtering training data for "good outcomes" taught a model the bad habits anyway — r/ArtificialInteligence
  5. DeepSeek offers user refunds after suffering another major outage — South China Morning Post Tech
  6. Point me to a good video on current state of the Chinese AI industry. [D] — r/MachineLearning
  7. Western and Chinese AI search engines cite largely separate webs. Qwen's sources were 96% Chinese even with half the questions in English — r/artificial
  8. Qwen3.8 Flash Next Game development — r/LocalLLM

Get the daily brief of stories like this at 6:30 every morning →