A transformer block is six tensors and a bus: read it like a pipeline
This story is from 2026-10-04. It is preserved in the archive; the latest stories are on the live feed.
Originally published at cchinchilla.dev . Part 5 of From code to weights, a 12-part series on ML fundamentals for engineers. Part 4 ended on one stage. A decoder is a stack of blocks, twelve in GPT-2 small, and attention is one of two stages in each. This post reads a whole block the way you'd read…
Read the full story at DEV Community — Machine Learning ↗
Timeline · 1 report
- 2026-10-04 16:00 · DEV Community — Machine Learning
A transformer block is six tensors and a bus: read it like a pipeline