The Honest Gradient Through a Binary Bottleneck Is Not Small. It Is Exactly Zero on 42 of 42 Coordinates.
This story is from 2026-08-29. It is preserved in the archive; the latest stories are on the live feed.
The bottleneck is six binary units, h = sign(pre) . The task is built so that a discrete bottleneck is exactly the right model: the true conditional depends on x only through m binary features, so a zero-excess student provably exists , the teacher scores exactly 0, and the score is a mean KL above…
Read the full story at DEV Community — Machine Learning ↗
Timeline · 1 report
- 2026-08-29 20:24 · DEV Community — Machine Learning
The Honest Gradient Through a Binary Bottleneck Is Not Small. It Is Exactly Zero on 42 of 42 Coordinates.