LSTM-Style Looped Transformers with Shared KV Caches
This story is from 2026-10-27. It is preserved in the archive; the latest stories are on the live feed.
arXiv:2610.33360v1 Announce Type: new Abstract: Recurrent transformers reuse weights to perform deep computation with fewer parameters. We study an LSTM-style transformer that also shares one key--value cache across its recurrent core. At width 1024, it uses 40.4% fewer parameters than an untied co…
Read the full story at arXiv cs.LG ↗
Timeline · 1 report
- 2026-10-27 04:00 · arXiv cs.LG
LSTM-Style Looped Transformers with Shared KV Caches