Stanford and UC Berkeley researchers release TokenGate, learned token pruning for long-context prefill (up to 11.3x faster TTFT)
TokenGate is a small drop-in module that adds trainable token pruning to existing LLMs. A gate at an early layer keeps the top-K document tokens. Layers above it process only those tokens, cutting prefill compute and KV cache. All numbers below are self-reported in the paper. What it does Scores ea…
Read the full story at r/machinelearningnews ↗
Timeline · 2 reports
- 2026-10-26 06:06 · r/LocalLLaMA
Researchers from Stanford and UC Berkeley Introduce TokenGate: Token-Gated Attention Makes Long-Context Prefill Up to 11.3x Faster - 2026-10-25 12:17 · r/machinelearningnews
Stanford and UC Berkeley researchers release TokenGate, learned token pruning for long-context prefill (up to 11.3x faster TTFT)