New DPO Variant Adds Reliability Weights to Handle Noisy Preference Data
A new paper proposes Reliability-Weighted Direct Preference Optimization, which penalizes unreliable comparisons instead of treating all preference pairs as correct, addressing reversed, tied, or noisy labels in preference data.
Read the full story at arXiv stat.ML ↗
Timeline · 1 report
- 2026-11-02 05:00 · arXiv stat.ML
Reliability-Weighted Direct Preference Optimization: Penalized Weights for Robust Preference Learning