Distributed LLM Inference via Communication-Efficient Speculative Decoding
arXiv:2610.35957v1 Announce Type: new Abstract: Distributed speculative decoding has emerged as a promising approach for device--server collaborative large language model (LLM) inference, but its decoding speedup is often limited by the vocabulary-scale communication overhead. In existing distribut…
Read the full story at arXiv cs.LG ↗
Timeline · 1 report
- 2026-10-28 04:00 · arXiv cs.LG
Distributed LLM Inference via Communication-Efficient Speculative Decoding