what's the best way of hosting qwen3.8-27b on 5090 in win11?
i use qwen3_8_27b_nvfp4-obl-v3.ninfer and i do coding with pi agent, i tried many configs/temps/parameters but model often keeps looping. --max-context 170840 ^ --kv-dtype int8 ^ --kv-capacity auto ^ --max-concurrency 2 ^ --prefill-chunk 4096 ^ --spec mtp ^ --draft-tokens 3 ^ --lm-head-draft ^ --vi…
Read the full story at r/LocalLLM ↗
Timeline · 1 report
- 2026-10-27 12:07 · r/LocalLLM
what's the best way of hosting qwen3.8-27b on 5090 in win11?