Echo: Error-Compensated 4-Bit Tensor-Parallel Communication for LLM Training and Inference
arXiv:2610.14277v1 Announce Type: new Abstract: Tensor parallelism ends every layer with an all-reduce that sits between two dependent matrix multiplications, so its cost cannot be hidden and grows as compute outpaces interconnect bandwidth. Sending 4-bit instead of 16-bit values would cut this cos…
Read the full story at arXiv cs.AI ↗
Timeline · 1 report
- 2026-10-14 04:00 · arXiv cs.AI
Echo: Error-Compensated 4-Bit Tensor-Parallel Communication for LLM Training and Inference