Rethinking Vision Architectures with Gated Linear Attention and KAN
arXiv:2609.22506v1 Announce Type: new Abstract: Vision Transformers allocate most parameters to multi-layer perceptrons (MLPs) for channel mixing, while token interactions usually rely on quadratic multi-head self-attention (MHSA). Linear attention reduces sequence complexity to O(N), but remains c…
Read the full story at arXiv cs.CV ↗
Timeline · 1 report
- 2026-09-22 04:00 · arXiv cs.CV
Rethinking Vision Architectures with Gated Linear Attention and KAN