Reveal the Mystery of DPO: The Connection between DPO and RL Algorithms
Xuerui Su, Yue Wang, Jinhua Zhu, Mingyang Yi, Feng Xu, Zhiming Ma, Yuting Liu · 2025
Read Reveal the Mystery of DPO: The Connection between DPO and RL Algorithms by Xuerui Su, Yue Wang, Jinhua Zhu, Mingyang Yi, Feng Xu, Zhiming Ma, Yuting Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-03095.txt.
Description
Xuerui Su and co-authors probe in 2025 the tie between a direct taste tune method and classic reward based learning for large models. Taste tune tools split into two camps: those that train a reward score, and those that skip it. The skip camp's odd loss form raised doubts on its class.
Three points frame the probe: how the loss is built, where it lands, and what parts drive its pace. A shared frame, named in the work, links all the kin tools by loss form. In that frame the end states of each tool come clear. Part tests then show what sets the meet pace.
Catalog record
| CREATOR | Xuerui Su, Yue Wang, Jinhua Zhu, Mingyang Yi, Feng Xu, Zhiming Ma, Yuting Liu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 177 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-03095