AINewsnow

Muir: Linear Attention with Sensitivity-Allocated Gating and Matched-Filter Readout (Google Research, 10M tokens in 25MB state) [R]

Paper: https://arxiv.org/abs/2611.08376 | HF: https://huggingface.co/papers/2611.08376 At 10M tokens, a 1.3B Transformer needs 2.1 TB of KV cache (0.32 TB even at 2.5-bit quantization). Channel-wise gated delta models (like Gated DeltaNet-2 / GDN-2) compress history into a fixed 25 MB state, but hi…

Read the full story at r/MachineLearning ↗

Timeline · 1 report

  1. 2026-11-06 19:10 · r/MachineLearning
    Muir: Linear Attention with Sensitivity-Allocated Gating and Matched-Filter Readout (Google Research, 10M tokens in 25MB state) [R]

More stories

  1. Claude Code drives wave of community-built apps, tools and workflows — DEV Community — AI
  2. Google Plans to Spend $17 Billion on Wyoming Data Center — Bloomberg AI
  3. Generate longer AI videos with Omni 1.2 Flash in Google Vids at no cost. — Google: The Keyword
  4. AI Daily Digest November 6, 2026: OpenAI Ships GPT-6 Cyber, Meta Buys Conductor, Anthropic Locks Down Claude Enterprise — DEV Community — AI
  5. Big Tech and green groups in talks over US data center guardrails — Economic Times Tech
  6. Parents sue Google, alleging Gemini on school Chromebook helped teen plan suicide — The Hill Technology
  7. Parents Sue Google Over AI Safety Risks on School Chromebooks — New York Times Technology
  8. Hinton Warns AI Could Disrupt Power and Finance: He Calls for Independent Testing Before Release — DEV Community — AI

Get the daily brief of stories like this at 6:30 every morning →