WavePrune: One period is often enough for RoPE
arXiv:2610.06963v1 Announce Type: new Abstract: Rotary Position Embedding (RoPE) encodes token positions by rotating each two-dimensional channel of the query and key vectors at a channel-specific frequency, making the attention logits invariant to a common shift of positions. However, this rotatio…
Read the full story at arXiv cs.CL ↗
Timeline · 1 report
- 2026-10-07 04:00 · arXiv cs.CL
WavePrune: One period is often enough for RoPE