voidinfer - near BF16 Qwen3.8-27B +-400k shared context on 1x RTX 5090 Windows
TL;DR: Compared to NInfer: 6× closer to BF16, up to 3.7× more context, about 25% slower decode. https://github.com/The8Darkness/voidinfer I have seen a lot of engines popping up that are all focusing on speed and sacrificing quality for it. I have been working on getting the highest possible qualit…
Read the full story at r/LocalLLM ↗
Timeline · 1 report
- 2026-10-22 21:30 · r/LocalLLM
voidinfer - near BF16 Qwen3.8-27B +-400k shared context on 1x RTX 5090 Windows