SpecRelay: Amortize or Avoid the Round Trip in Edge-Cloud Speculative Decoding
This story is from 2026-11-18. It is preserved in the archive; the latest stories are on the live feed.
arXiv:2611.18963v1 Announce Type: new Abstract: Edge-cloud speculative decoding moves autoregressive drafting to an edge device while a larger cloud model verifies draft tokens in parallel, so that edge drafting and cloud verification overlap under a synchronous draft-verify schedule. Beyond low-la…
Read the full story at arXiv cs.AI ↗
Timeline · 1 report
- 2026-11-18 05:00 · arXiv cs.AI
SpecRelay: Amortize or Avoid the Round Trip in Edge-Cloud Speculative Decoding