Latent Preference Coding: Aligning Large Language Models via Discrete Latent Codes

Zhuocheng Gong, Jian Guan, Wei Wu, Huishuai Zhang, Dongyan Zhao · 2025

Read Latent Preference Coding: Aligning Large Language Models via Discrete Latent Codes by Zhuocheng Gong, Jian Guan, Wei Wu, Huishuai Zhang, Dongyan Zhao (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-04993.txt.

Description

Zhuocheng Gong, Jian Guan, Wei Wu, Huishuai Zhang, and Dongyan Zhao introduce Latent Preference Coding, a way to align large language models with human preferences. Their 2025 paper notes that preferences are intricate and can pull in conflicting directions.

Most methods rely on an explicit or implicit reward function, which flattens that complexity. Their framework models hidden factors behind holistic preferences, and their combinations, as discrete latent codes. It infers the factors and their importance from data, with no predefined reward function or hand-set weights. The method joins smoothly with various offline alignment algorithms. Tests on multiple benchmarks show consistent gains over prior approaches.

Catalog record

CREATORZhuocheng Gong, Jian Guan, Wei Wu, Huishuai Zhang, Dongyan Zhao
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE48 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-04993

Browse the full library · About the Tower of Babel