Canonical-basis realignment for Transformer LLMs: every hidden axis becomes independently measurable and controllable
This story is from 2026-08-29. It is preserved in the archive; the latest stories are on the live feed.
The code essentially gives you a way to rotate a Transformer's internal coordinate system into a canonical basis that aligns with its own weight matrices in a lossless way. By absorbing the normalization gains directly into the adjacent weights and using orthogonal matrices built from the singular…
Read the full story at Lobsters AI ↗
Timeline · 1 report
- 2026-08-29 20:16 · Lobsters AI
Canonical-basis realignment for Transformer LLMs: every hidden axis becomes independently measurable and controllable