AINewsnow

HauHauCS 27B Uncensored port to Ninfer-4090

Working this tonight. 262K context instead of 131K, and about 130 t/s decode instead of the 91.6 t/s measured on the current IQ4_XS entry, plus NInfer's faster INT8 prefill. NInfer quantizes to about 17 GB, roughly the same size as the IQ4_XS, and going through Q8_0 first adds almost no error. CPU…

Read the full story at r/LocalLLM ↗

Timeline · 1 report

  1. 2026-10-07 21:08 · r/LocalLLM
    HauHauCS 27B Uncensored port to Ninfer-4090

More stories

  1. GPT-6 and Intelligent UI for everyone — OpenAI News
  2. Introducing Claude Haiku 5.5 on AWS — AWS Machine Learning Blog
  3. Introducing Mistral Large 4 — Mistral AI News
  4. Sharing AI progress in mathematics — OpenAI News
  5. Mistral Says Its New AI Model ‘Le Chonk’ Is the Best Open-Weight Offering Outside of China — Wired AI
  6. NVIDIA, Microsoft Kick Off a New Beginning for Windows PCs With RTX Spark and AI Agents — NVIDIA Blog
  7. Introducing Playground: Create and play custom games — Google AI Blog
  8. OpenAI Decisions API now available on AI Gateway — Vercel Blog

Get the daily brief of stories like this at 6:30 every morning →