LazyFlash: Forward-Dense, Backward-Sparse Attention for Accelerated Long-Context Fine-Tuning
arXiv:2610.31911v1 Announce Type: new Abstract: As large language models (LLMs) process increasingly long sequences, attention becomes a major computational bottleneck. Sparse attention mostly targets inference, yet many methods still train densely or cannot support sparse training, and their added…
Read the full story at arXiv cs.CL ↗
Timeline · 1 report
- 2026-10-26 04:00 · arXiv cs.CL
LazyFlash: Forward-Dense, Backward-Sparse Attention for Accelerated Long-Context Fine-Tuning