Demystifying the Unreasonable Effectiveness of Online Alignment Methods
Enoch Hyunwook Kang · 2026
Read Demystifying the Unreasonable Effectiveness of Online Alignment Methods by Enoch Hyunwook Kang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2604-17207.txt.
Description
Kang argues that greedy alignment methods outperform existing regret guarantees, because KL regret and reward learning are different targets. KL regret tracks recovery of the aligned policy, while practitioners often only need the reward inducing the correct best response. Isolating reward learning via temperature-zero regret, which scores only the top response, Kang proves reference-logged offline RLHF reaches exponentially small expected regret, needing only logarithmically many comparisons for regret at most epsilon. Greedy online RLHF achieves bounded temperature-zero regret, extending to its pairwise online DPO form.
Catalog record
| CREATOR | Enoch Hyunwook Kang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 110567 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2604.17207