You fine-tuned at full precision. You serve in FP8. Those were two different models.
transformers v5.22.0 made FP8 models trainable with LoRA. The interesting part is the workflow it removes, not the feature it adds. You fine-tuned at full precision. You serve in FP8. Those were two different models. The line From the transformers v5.22.0 release notes, 29 October 2026: FP8 quantiz…
Read the full story at DEV Community — Machine Learning ↗
Timeline · 1 report
- 2026-11-12 08:02 · DEV Community — Machine Learning
You fine-tuned at full precision. You serve in FP8. Those were two different models.