Preference Robustness for DPO with Applications to Public Health

Cheol Woo Kim, Shresth Verma, Mauricio Tec, Milind Tambe · 2025

Read Preference Robustness for DPO with Applications to Public Health by Cheol Woo Kim, Shresth Verma, Mauricio Tec, Milind Tambe (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2509-02709.txt.

Description

Cheol Woo Kim and three co-authors wrote a 2025 AI alignment paper. It tunes language models from human preferences. The preferences are noisy, and the stakes are public health. Their testbed designs reward functions that guide sequential resource allocation.

The proposed DPO-PRO algorithm builds on Direct Preference Optimization. It adds a distributionally robust layer. That layer allows for uncertainty about how preferences are distributed. Unlike earlier robust versions, it stays only mildly conservative. Evaluation uses a real maternal mobile health programme. The nonprofit ARMMAN runs it. Standard alignment benchmarks are also used. The method improved robustness to noisy preference signals over rival variants. It matched a self-reflection baseline on reward design, at far lower run-time cost.

Catalog record

CREATORCheol Woo Kim, Shresth Verma, Mauricio Tec, Milind Tambe
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE58 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2509-02709

Browse the full library · About the Tower of Babel