Recurrent Looped Transformer
This story is from 2026-09-16. It is preserved in the archive; the latest stories are on the live feed.
Recurrent Looped Transformer (RLT) passes the decoder's final hidden state to the next token, together with that token's causal encoder representation. The decoder reads encoder-derived global KV memory and maintains a sliding-window attention (SWA) cache at every layer. The same update runs over p…
Read the full story at r/LocalLLaMA ↗
Timeline · 1 report
- 2026-09-16 13:30 · r/LocalLLaMA
Recurrent Looped Transformer