A transformer block is six tensors and a bus: read it like a pipeline
Originally published at cchinchilla.dev . Part 5 of From code to weights, a 12-part series on ML fundamentals for engineers. Part 4 ended on one stage. A decoder is a stack of blocks, twelve in GPT-2 small, and attention is one of two stages in each. This post reads a whole block the way you'd read…
Read the full story at DEV Community — Machine Learning ↗
Timeline · 1 report
- 2026-10-04 16:00 · DEV Community — Machine Learning
A transformer block is six tensors and a bus: read it like a pipeline