AINewsnow

Open-source method (MACaQ) beats Unsloth GGUFs!

Some results for those using GGUFs and llama.cpp: GGUF quality: MACaQ vs. Unsloth and Bartowski Qwen3-4B (IQ4_XS) Quantization GGUF size Mean KL ↓ MACaQ 2.260 GB 0.02804 Bartowski 2.271 GB 0.03443 Unsloth 2.271 GB 0.03663 23.5% lower KL divergence than Unsloth and 18.6% lower than Bartowski , at vi…

Read the full story at r/LocalLLaMA ↗

Timeline · 1 report

  1. 2026-11-11 17:32 · r/LocalLLaMA
    Open-source method (MACaQ) beats Unsloth GGUFs!

More stories

  1. Local LLMs for Agentic Coding: A Practical Guide for Modest Hardware — DEV Community — AI
  2. Kinetic: a new open-source LLM engine in Rust. Runs your GGUFs on Windows and Linux, replaces llama-server with the same flags — r/LocalLLM
  3. Caution about fast inference engines — r/LocalLLM
  4. Run Gemma 4 31B-it Uncensored on a 12GB GPU: Complete Local Vision & 256K Context Guide with llama.cpp, DeepSeek Harness & Hermes Agent — DEV Community — Machine Learning
  5. llama.cpp: Add FP8 KV-cache — r/LocalLLaMA
  6. Study examines whether large language models internally represent psychological constructs — arXiv cs.AI
  7. How do you fine-tune Llama on your own data in the cloud? — DEV Community — Machine Learning
  8. Your coding questions are exactly where AI models disagree most — DEV Community — AI

Get the daily brief of stories like this at 6:30 every morning →