Random Is Hard to Beat: Active Selection in online DPO with Modern LLMs

Giyeong Oh Junghyun Lee Jaehyun Park Youngjae Yu Wonho Bae Junhyug Noh, [1.0em] Seoul National University · 2026

Read Random Is Hard to Beat: Active Selection in online DPO with Modern LLMs by Giyeong Oh Junghyun Lee Jaehyun Park Youngjae Yu Wonho Bae Junhyug Noh, [1.0em] Seoul National University (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-02766.txt.

Description

Oh and five colleagues test active preference learning against random sampling for online direct preference optimization with modern language models. Uncertainty-based selection yields negligible gains in proxy win rates over random sampling across harmlessness, helpfulness, and instruction-following settings. Worse, win rates can rise while general capability falls, and active selection fails to prevent that collapse or cut variance much better than random sampling.

Catalog record

CREATORGiyeong Oh Junghyun Lee Jaehyun Park Youngjae Yu Wonho Bae Junhyug Noh, [1.0em] Seoul National University
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36485
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2604.02766

Browse the full library · About the Tower of Babel