Applying Sliding Window Attention to pretrained LLMs at inference time [P]
This story is from 2026-09-06. It is preserved in the archive; the latest stories are on the live feed.
I've been working on a practical implementation of **Sliding Window Attention (SWA)** for pretrained Hugging Face causal LLMs. The idea is simple: instead of allowing every generated token to attend to the complete historical KV cache, maintain a bounded cache consisting of: **attention sinks + rec…
Read the full story at r/machinelearningnews ↗
Timeline · 1 report
- 2026-09-06 21:06 · r/machinelearningnews
Applying Sliding Window Attention to pretrained LLMs at inference time [P]