AINewsnow

I built ALHR: A tree based sparse attention system that achieves sub-quadratic inference while retaining accuracy. [P]

ALHR - Adaptive Learnable Hierarchical Routing, uses static binary trees and learnable functions to minimize the amount of keys to be read. It does use a dense teacher while phase 1 of training however. MQAR TEST AT 1024 TOKENS - Average keys read per query by dense - 512 Keys Average keys read per…

Read the full story at r/MachineLearning ↗

Timeline · 1 report

  1. 2026-10-09 13:29 · r/MachineLearning
    I built ALHR: A tree based sparse attention system that achieves sub-quadratic inference while retaining accuracy. [P]

More stories

  1. GPT-6 and Intelligent UI for everyone — OpenAI News
  2. Introducing Claude Haiku 5.5 on AWS — AWS Machine Learning Blog
  3. NVIDIA, Microsoft Kick Off a New Beginning for Windows PCs With RTX Spark and AI Agents — NVIDIA Blog
  4. OpenAI Decisions API now available on AI Gateway — Vercel Blog
  5. Introducing Playground: Create and play custom games — Google AI Blog
  6. Anthropic bans ‘abusive or cruel behavior’ toward Claude — The Verge AI
  7. Impactful scheduling for GPU clusters — Allen Institute for AI (Ai2)
  8. Sophos cuts threat investigation time by 96% with OpenAI Daybreak — OpenAI News

Get the daily brief of stories like this at 6:30 every morning →