AINewsnow

Run GLM-5.3-Flash (321B) locally with Project Maya and EXL3 on one RTX 5090

This story is from 2026-10-16. It is preserved in the archive; the latest stories are on the live feed.

GLM-5.3-Flash is a 321B mixture-of-experts model, and its 3.05 bpw EXL3 quant is 125 GB, too big to fit in my workstation's 128 GB of RAM next to the OS. It still runs locally here: on one RTX 5090 with 128 GB of RAM, with a 512K-token context, at about 25 tokens per second. This guide shows how, w…

Read the full story at DEV Community — AI ↗

Timeline · 1 report

  1. 2026-10-16 14:33 · DEV Community — AI
    Run GLM-5.3-Flash (321B) locally with Project Maya and EXL3 on one RTX 5090

More stories

  1. Update-Aware Reasoning in GLM-5.3: Training & Alignment Faking — r/singularity
  2. Z.ai’s GLM-5.3 raises researcher fears the Chinese model can fake AI safety tests — South China Morning Post Tech
  3. CPU Inference performance for GLM5.3-Flash and Qwen3.8? — r/LocalLLM
  4. Instead of waiting for an M5 Ultra 512 GB: pairing a new M5 Ultra 256 GB with my M3 Ultra 512 GB for disaggregated prefill/decode (GLM-5.3-Flash, MLX, Thunderbolt 5 RDMA) — r/LocalLLaMA
  5. GLM-5.3-Flash pruned to half the experts for coding: Q3 (71 GB) solves every task the full 341 GB model solves, IQ2_XXS fits 24 GB VRAM + 32 GB RAM — r/LocalLLM
  6. Users evaluate Qwen 3.8 Flash Next performance and quantization options — r/LocalLLM
  7. Meta and Apple collaborate to enable Muse Spark Open AI model on Macs — Wall Street Journal Technology
  8. Reflection AI releases Beam, a 501B open-weight model for agentic coding — AWS Machine Learning Blog

Get the daily brief of stories like this at 6:30 every morning →