Taco: cross-platform LLM compression that doesn't break when you change your GPU or CPU. [P]
The problem: an LLM compressor needs the decoder to compute exactly the same probabilities as the encoder. Floats differ across hardware; float GPT-2 encoded on a Mac decoded 0 of 4 windows on x86. The approach: integer-only inference (int8 weights, fixed rounding, lookup tables); an exact float32…
Read the full story at r/MachineLearning ↗
Timeline · 2 reports
- 2026-10-19 20:36 · r/huggingface
Taco: cross-platform LLM compression that doesn't break when you change your GPU or CPU. - 2026-10-19 20:05 · r/MachineLearning
Taco: cross-platform LLM compression that doesn't break when you change your GPU or CPU. [P]