Robust Multi-Objective Controlled Decoding of Large Language Models
Seongho Son, William Bankes *, Sangwoong Yoon *, Shyam Sundhar Ramesh *, Xiaohang Tang, Ilija Bogunovic · 2025
Read Robust Multi-Objective Controlled Decoding of Large Language Models by Seongho Son, William Bankes *, Sangwoong Yoon *, Shyam Sundhar Ramesh *, Xiaohang Tang, Ilija Bogunovic (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2503-08796.txt.
Description
Son and a team of colleagues introduced a decoding method in twenty twenty-five that aligns language models to many human goals at once. Goals such as helpfulness and safety can conflict. Standard controlled decoding balances average rewards, which can leave the worst goal poorly served.
Their method, RMOD, instead maximizes the worst-case reward across goals. They frame the problem as a two-player game between hostile reward weights and the sampling policy, solved at a Nash equilibrium. That game reduces to a convex problem for the worst weights, with the best policy derived directly. A practical version adds little computing cost over standard decoding. Tests with up to ten objectives on alignment datasets beat baselines on worst-case rewards and win rates, they report.
Catalog record
| CREATOR | Seongho Son, William Bankes *, Sangwoong Yoon *, Shyam Sundhar Ramesh *, Xiaohang Tang, Ilija Bogunovic |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 77 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2503-08796