Reveal the Mystery of DPO: The Connection between DPO and RL Algorithms

Xuerui Su, Yue Wang, Jinhua Zhu, Mingyang Yi, Feng Xu, Zhiming Ma, Yuting Liu · 2025

Read Reveal the Mystery of DPO: The Connection between DPO and RL Algorithms by Xuerui Su, Yue Wang, Jinhua Zhu, Mingyang Yi, Feng Xu, Zhiming Ma, Yuting Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-03095.txt.

Description

Xuerui Su and co-authors probe in 2025 the tie between a direct taste tune method and classic reward based learning for large models. Taste tune tools split into two camps: those that train a reward score, and those that skip it. The skip camp's odd loss form raised doubts on its class.

Three points frame the probe: how the loss is built, where it lands, and what parts drive its pace. A shared frame, named in the work, links all the kin tools by loss form. In that frame the end states of each tool come clear. Part tests then show what sets the meet pace.

Catalog record

CREATORXuerui Su, Yue Wang, Jinhua Zhu, Mingyang Yi, Feng Xu, Zhiming Ma, Yuting Liu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE177 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-03095

Browse the full library · About the Tower of Babel