Instead of waiting for an M5 Ultra 512 GB: pairing a new M5 Ultra 256 GB with my M3 Ultra 512 GB for disaggregated prefill/decode (GLM-5.3-Flash, MLX, Thunderbolt 5 RDMA)
The problem I'm solving: I have an M3 Ultra 512 GB and the memory is still the reason to own it, but prefill on long prompts is slow. A 17.5k-token prompt takes 41 s before the first token. The M5 Ultra fixes prefill but ships with 256 GB max right now, and an M5 Ultra 512 GB isn't a thing yet. So…
Read the full story at r/LocalLLaMA ↗