Same quantized model, two different fates: one chip refused it, the other ran it in 1.39 ms
I took one quantized NanoDet model and offered it to two Qualcomm devices. One refused to load it at all. The other ran all 324 layers on the NPU in 1.39 ms. The model was identical. The quantization was identical. The difference was a float tail the quantizer couldn't touch — and whether the devic…
Read the full story at DEV Community — Machine Learning ↗
Timeline · 1 report
- 2026-11-12 01:41 · DEV Community — Machine Learning
Same quantized model, two different fates: one chip refused it, the other ran it in 1.39 ms