Zyphra AI Introduces Hierarchical Attention for Subquadratic Attention in Frozen LLMs

Zyphra AI introduced Zyphra Hierarchical Attention (ZHA), a tree-indexed sparse attention method that retrofits frozen LLMs using 4B tokens of distillation, matching dense attention on reasoning and 256K retrieval benchmarks.

Read the full story at MarkTechPost ↗

Timeline · 1 report

2026-11-03 01:09 · MarkTechPost
Zyphra AI Introduces Zyphra Hierarchical Attention (ZHA): A Path to True Subquadratic Attention

More stories

OpenAI appoints former US Army Secretary Dan Driscoll to national security role — OpenAI News
Anthropic reports $17B quarterly revenue, mounting losses and spending commitments ahead of IPO — Reuters AI
Marvell to acquire Lumentum in $115 billion all-stock AI data-centre deal — DigiTimes Asia
OpenAI to merge ChatGPT's Chat and Work tabs into a single desktop experience — TechCrunch AI
Apple CEO John Ternus says company won't ship AI for AI's sake — Techmeme
Anthropic confirms new Claude Code limits to prevent recreating existing video games — Engadget
OpenAI launches Grid Protectors Program to cut data center power use during grid strain — OpenAI News
Israeli startup 42AI exits stealth with $62.5M seed for behavioral world model — TechCrunch AI

Get the daily brief of stories like this at 6:30 every morning →