AINewsnow

A 30B MoE model was running at 6.7 s/token on two V100s. The cause was not the kernels.

I'm the author of NeuroBrix, an open-source (Apache 2.0) inference engine that runs models without PyTorch at runtime. I want to share a debugging story from this week, because the cause surprised me. Setup: Qwen3-Coder-30B-A3B (mixture of experts) on two Tesla V100 32GB. Through the serving daemon…

Read the full story at r/machinelearningnews ↗

Timeline · 1 report

  1. 2026-11-11 23:26 · r/machinelearningnews
    A 30B MoE model was running at 6.7 s/token on two V100s. The cause was not the kernels.

More stories

  1. Tesla to add production shift to clear Cybertruck backlog ahead of phase-out — DigiTimes Asia
  2. Cathie Wood's ARK Invest Says Tesla's Robotaxi Cash Flow Could Bankroll SpaceX's AI Ambitions, Backing a Potential Merger of Elon Musk's Two Companies. Is This the Thesis That Finally Reprices Tesla Stock? — The Globe and Mail
  3. I don't think engineers will exist in a few years — DEV Community — AI
  4. The phantom life of Alice Huang, alleged Chinese spy who had affair with Nvidia CEO — CSET Georgetown
  5. Anthropic IPO Reportedly Oversubscribed as AI Trade Rebounds — CNBC Technology
  6. Google launches Gemini Decisions API with Gemini Decision 1 model — Google Cloud AI Blog
  7. AWS launches local emulator simulating 200+ services, written entirely by AI agents — AWS Machine Learning Blog
  8. Google DeepMind launches AlphaGenome 2 to predict DNA variant effects across cell types — Google DeepMind Blog

Get the daily brief of stories like this at 6:30 every morning →