Watching attention flow while an LLM generates tokens
This story is from 2026-09-05. It is preserved in the archive; the latest stories are on the live feed.
https://reddit.com/link/1w81p3p/video/50d4gyg3lpnh1/player I made a short animation of how attention moves while a Transformer generates text. First few seconds are prefill: the model reads the whole prompt, tokens looking at earlier ones in parallel. After that is decode: one token at a time. Each…
Read the full story at r/deeplearning ↗
Timeline · 2 reports
- 2026-09-06 08:27 · r/learnmachinelearning
Watching attention flow while an LLM generates tokens - 2026-09-05 14:06 · r/deeplearning
Watching attention flow while an LLM generates tokens