Latent Preference Coding: Aligning Large Language Models via Discrete Latent Codes
Zhuocheng Gong, Jian Guan, Wei Wu, Huishuai Zhang, Dongyan Zhao · 2025
Read Latent Preference Coding: Aligning Large Language Models via Discrete Latent Codes by Zhuocheng Gong, Jian Guan, Wei Wu, Huishuai Zhang, Dongyan Zhao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-04993.txt.
Description
Zhuocheng Gong, Jian Guan, Wei Wu, Huishuai Zhang, and Dongyan Zhao introduce Latent Preference Coding, a way to align large language models with human preferences. Their 2025 paper notes that preferences are intricate and can pull in conflicting directions.
Most methods rely on an explicit or implicit reward function, which flattens that complexity. Their framework models hidden factors behind holistic preferences, and their combinations, as discrete latent codes. It infers the factors and their importance from data, with no predefined reward function or hand-set weights. The method joins smoothly with various offline alignment algorithms. Tests on multiple benchmarks show consistent gains over prior approaches.
Catalog record
| CREATOR | Zhuocheng Gong, Jian Guan, Wei Wu, Huishuai Zhang, Dongyan Zhao |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 48 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-04993