AINewsnow

I expanded FreeToken's GGUF support to Qwen MoE/dense, 1-4 bit K/I quants and sharded GGUFs. Tested 35B Ornith at 47-52 tok/s on an 8GB RTX 4060 laptop

This story is from 2026-08-24. It is preserved in the archive; the latest stories are on the live feed.

I've been messing with FreeToken since the release because the idea behind it immediately caught my attention. Getting 35B-class MoE models running interactively on an 8GB laptop GPU is already pretty wild. The problem for me was that the initial GGUF path was much narrower than the GGUF ecosystem…

Read the full story at r/LocalLLM ↗

Timeline · 2 reports

  1. 2026-08-24 19:35 · r/huggingface
    I expanded FreeToken's GGUF support to Qwen MoE/dense, 1-4 bit K/I quants and sharded GGUFs. Tested 35B Ornith at 47-52 tok/s on an 8GB RTX 4060 laptop
  2. 2026-08-24 19:32 · r/LocalLLM
    I expanded FreeToken's GGUF support to Qwen MoE/dense, 1-4 bit K/I quants and sharded GGUFs. Tested 35B Ornith at 47-52 tok/s on an 8GB RTX 4060 laptop

More stories

  1. Alibaba ships Qwen3.8-Omni-Flash to watch, listen and call tools — r/LocalLLM
  2. Qwen Image 2.1 PR to ComfyUI — r/StableDiffusion
  3. Alibaba releases Qwen-Image-2.1, a 7B open-weight model it says outperforms most closed-source models, with native transparency and up to ten reference images (Qwen) — Techmeme
  4. Qwen q4 3.8 27b 16 tok/s 32k RTX 3060 :D — r/LocalLLM
  5. 10 hours left fo Qwen Image 2.1 Public Open Source Release — r/StableDiffusion
  6. Qwen 3.8 27B running on a single RTX 5090 researches and creates a full animation using only code. — r/artificial
  7. US government website used Chinese model the FBI called "malicious" — Ars Technica AI
  8. M2 Mac ultra128gb Qwen flash next — r/LocalLLM

Get the daily brief of stories like this at 6:30 every morning →