Nearly Optimal Active Preference Learning and Its Application to LLM Alignment
Yao Zhao, Kwang-Sung Jun · 2026
Read Nearly Optimal Active Preference Learning and Its Application to LLM Alignment by Yao Zhao, Kwang-Sung Jun (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-01581.txt.
Description
Yao Zhao and Kwang Sung Jun cut the cost of aligning language models by choosing which preference labels to collect next. Active learning designs borrow optimality rules ignoring the special shape of preference data, so the authors question their fit here. They propose two methods: one with the first instance dependent label complexity guarantee here, and a simpler greedy method. They prove the key term optimal up to a constant tied to the logit bound, and tests on real and synthetic data show better sample efficiency.
Catalog record
| CREATOR | Yao Zhao, Kwang-Sung Jun |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 69305 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602.01581