Blackwell quantisation on my own card
This story is from 2026-10-28. It is preserved in the archive; the latest stories are on the live feed.
Quantising a small model down from FP16 to Q4_K_M on my RTX 5070 Ti more than doubled throughput and cut VRAM by two thirds, with output that stayed 98% similar to the full-precision baseline by embedding distance. Both registered gates passed. The interesting part turned out to be where the remain…
Read the full story at DEV Community — AI ↗
Timeline · 1 report
- 2026-10-28 06:13 · DEV Community — AI
Blackwell quantisation on my own card