New paper proposes tensor-rank theory for tied and untied MLP blocks in transformers
An arXiv paper introduces CP-MLPs, a tensor-rank framework analyzing transformer feed-forward design choices including squared activations, gating, and tied versus untied parameter sharing.
Read the full story at arXiv stat.ML ↗
Timeline · 1 report
- 2026-11-11 05:00 · arXiv stat.ML
CP-MLPs: A Tensor-Rank Theory of Tied and Untied MLP Blocks