At 32K-128K context, what prefill and decode rates are you getting per weight quant?
I read back through recent threads here before asking, and most benchmark posts report one configuration at one context size. That leaves the axis I care about unreadable: how throughput splits between prompt processing and generation as context grows. From the numbers people do post, prefill looks…
Read the full story at r/LocalLLM ↗
Timeline · 1 report
- 2026-10-13 16:31 · r/LocalLLM
At 32K-128K context, what prefill and decode rates are you getting per weight quant?