[P] FP8 on the wire for VLA training: what actually paid off on a PCIe-only 8-GPU box, and the 48%-fewer-bytes change that bought exactly 0 ms

We train a three-modality VLA model (video diffusion + action expert + a frozen VLM for understanding, mixture-of-transformers style). Originally on 8×A800 with NVLink; we're moving it to 8× workstation-class Blackwell cards, which means no NVLink and no NVSwitch — every GPU-to-GPU byte goes over P…

Read the full story at r/deeplearning ↗

Timeline · 1 report

1. 2026-09-20 13:28 · r/deeplearning
   [P] FP8 on the wire for VLA training: what actually paid off on a PCIe-only 8-GPU box, and the 48%-fewer-bytes change that bought exactly 0 ms