AINewsnow

GSQHalo.cpp - Another day, another fork. This one is for the memory constraint folks. 2×256K + MTP, 1386 t/s prefill and 44 t/s decode at 128K using GSQ-RCO Quants on a 96GB machine, KV cache storage on SSD

Coverage of "GSQHalo.cpp - Another day, another fork. This one is for the memory constraint folks. 2×256K + MTP, 1386 t/s prefill and 44 t/s decode at 128K using GSQ-RCO Quants on a 96GB machine, KV cache storage on SSD" from 1 source, with a live timeline of who reported what and when.

Read the full story at r/LocalLLM ↗

Timeline · 1 report

  1. 2026-10-04 00:09 · r/LocalLLM
    GSQHalo.cpp - Another day, another fork. This one is for the memory constraint folks. 2×256K + MTP, 1386 t/s prefill and 44 t/s decode at 128K using GSQ-RCO Quants on a 96GB machine, KV cache storage on SSD

More stories

  1. NVIDIA DGX Spark 64GB Gives Developers More Ways to Build and Scale Local AI — NVIDIA Blog
  2. Guided Vision in Gemini Live: built for accessibility — Google Gemini Blog
  3. Google tests its plan for AI data centers in space with Project Suncatcher — Scientific American
  4. A model guide for the GPT-6 family — OpenAI News
  5. The latest AI news we announced in September 2026 — Google AI Blog
  6. Introducing Olmo-core 3: Open, scalable training infrastructure for large MoEs — Allen Institute for AI (Ai2)
  7. OpenAI Fires Researchers for Allegedly Sharing Information with AI Safety Group — Wall Street Journal Technology
  8. Google unveils Gemini 4 Argon: Its most powerful AI model yet, focused on coding and cyber defence — Mint AI

Get the daily brief of stories like this at 6:30 every morning →