Basalt 0.4.0: More models, still as fast - Qwen 27B up to 770 tok/s on a 5090
Hi everyone! I wanted to post a quick update on Basalt . Since last I posted, multi model and router support was implemented, starting with DeepSeek V4.1 Flash (IQ1_M 90 tok/s conv, 246 tok/s struct on a 5090 + 5060 Ti). Also new: video support, files in chats, and a bunch of fixes and improvements…
Read the full story at r/LocalLLaMA ↗
Timeline · 1 report
- 2026-11-15 17:30 · r/LocalLLaMA
Basalt 0.4.0: More models, still as fast - Qwen 27B up to 770 tok/s on a 5090