Qwen3.8-27B at ~126 tok/s on 2× RTX 5060 Ti 16 GB SGLang + NVFP4 + DFlash2 (107 tok/s at 60K context, 216 tok/s on JSON)
Spent the last few nights benchmarking different inference engines on two 16 GB RTX 5060 Ti cards. This is the configuration I'm most happy with so far. TL;DR: Qwen3.8-27B with NVFP4, a DFlash2 drafter, and tensor parallelism (TP=2) achieves approximately 126 tok/s single-stream decode on two RTX 5…
Read the full story at r/LocalLLM ↗