Semantic-aware Wasserstein Policy Regularization for Large Language Model Alignment
Byeonghu Na, Hyungho Na, Yeongmin Kim, Suhyeon Jo, HeeSun Bae, Mina Kang, Il-Chul Moon · 2026
Read Semantic-aware Wasserstein Policy Regularization for Large Language Model Alignment by Byeonghu Na, Hyungho Na, Yeongmin Kim, Suhyeon Jo, HeeSun Bae, Mina Kang, Il-Chul Moon (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2602-01685.txt.
Description
Byeonghu Na and colleagues align language models with human preferences by fixing a gap in training. Training seeks reward under KL divergence penalty, but KL and f-divergence variants compare token chances at same indices, missing semantic similarity. Their Wasserstein Policy Regularization uses the entropy Wasserstein distance, which uses token space geometry. Its dual form turns the penalty into reward terms through dual variables, giving a usable objective for RL algorithms. Tests show it beats KL and f-divergence baselines, showing the value of semantic policy distances for alignment.
Catalog record
| CREATOR | Byeonghu Na, Hyungho Na, Yeongmin Kim, Suhyeon Jo, HeeSun Bae, Mina Kang, Il-Chul Moon |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 76965 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2602.01685