KLD is not the best metric out there and there isn't a single best one.
Following my previous post on how KLD is misunderstood , there were a few claims that KLD is the best metric you can use and some other comments were asking, "If we stop using KLD, what should we use instead?" I'm gonna explain both a bit more here. Assume you're a lab that does distillation. In th…
Read the full story at r/LocalLLaMA ↗
Timeline · 1 report
- 2026-10-26 16:52 · r/LocalLLaMA
KLD is not the best metric out there and there isn't a single best one.