AINewsnow

Basalt (NVFP4) vs Unsloth Studio (llama.cpp IQ4_XS) — 5090 with 64GB RAM

I wanted to share my experience and the brief testing I've done with Basalt and spread the word. This engine is shockingly fast, ESPECIALLY on my setup. It leverages Blackwell spec GPUs, that is the only catch here. If you don't have a 50 series GPU, you probably won't benefit. If you do, you'll ge…

Read the full story at r/LocalLLM ↗

Timeline · 1 report

  1. 2026-10-29 05:55 · r/LocalLLM
    Basalt (NVFP4) vs Unsloth Studio (llama.cpp IQ4_XS) — 5090 with 64GB RAM

More stories

  1. Learned attention gates separate instructions from data to resist prompt injection — DEV Community — Machine Learning
  2. Generate a training set with DeepSeek, then fine-tune a small model on a rented GPU — DEV Community — Machine Learning
  3. I tested LFM2.5-24B-A2B vs Qwen3.6-35B-A3B on 75 fresh multi-step tool tasks. Qwen solved 91%, LFM2.5 49%, and LFM2.5 said "done" on most of its failures — r/LocalLLM
  4. Gemma 3 1B on the Nexus 5X (Directly on the Linux Kernel, No Android) — 2 GB RAM, 4.47 t/s — r/LocalLLM
  5. Nanbeige4.3-33B Q4 on a MacBook Pro M5 Max, mlx-lm vs llama.cpp vs Kapitsa, Prefill/Decode 512 to 32K range. — r/LocalLLaMA
  6. A 2013 Mac Pro runs a 35B model at 66 tok/s reading. The first benchmark I trusted was wrong by about 9x. — r/LocalLLM
  7. mtmd: enable video input support for LFM3 model family by tdakhran · Pull Request #31283 · ggml-org/llama.cpp — r/LocalLLaMA
  8. shura.cpp: a llama.cpp fork with a VRAM governor, so a 35B MoE can share one 16 GB card with the desktop you keep using — r/LocalLLM

Get the daily brief of stories like this at 6:30 every morning →