vLLM 0.33 Adds a Lossy Speculative Verifier: Why Single-Run Accuracy Tables Cannot Tell You It Is Safe
This story is from 2026-11-05. It is preserved in the archive; the latest stories are on the live feed.
TL;DR What shipped: vLLM 0.33.0 (PyPI, 2026-11-02) includes FLy, an opt-in speculative decoding verifier that deliberately accepts some draft tokens the target model would have rejected. The PR states it "does not preserve the target distribution and may reduce output quality." Standard verificatio…
Read the full story at DEV Community — Machine Learning ↗
Timeline · 1 report
- 2026-11-05 14:05 · DEV Community — Machine Learning
vLLM 0.33 Adds a Lossy Speculative Verifier: Why Single-Run Accuracy Tables Cannot Tell You It Is Safe