SMat-Attention: Structured Long-Context Sequence Modeling
arXiv:2609.36062v1 Announce Type: new Abstract: Long-context sequence models face a fundamental tradeoff: softmax attention uses flexible token-level interactions at quadratic cost, whereas linear attention obtains linear-time training and constant-time decoding by compressing history into a fixed-…
Read the full story at arXiv cs.AI ↗
Timeline · 1 report
- 2026-09-30 04:00 · arXiv cs.AI
SMat-Attention: Structured Long-Context Sequence Modeling