Gyro-L (trellis-quantized Qwen3.8-Flash-Next) on Basalt: 262k context at ~50 tok/s on a 16 GB 5060 Ti + 64 GB RAM. PR looking for testers
I ported AgentionAI's Gyro-L quant of Qwen3.8-Flash-Next to Basalt, a CUDA inference engine built only for that model. Basalt runs on NVIDIA Blackwell only (RTX 50-series), on Linux or Windows. Gyro-L uses trellis-coded experts with a Hadamard rotation. Disclosure: most of the code and the measurem…
Read the full story at r/LocalLLM ↗