A Survey on Personalized and Pluralistic Preference Alignment in Large Language Models

Zhouhang Xie, Junda Wu, Yiran Shen, Yu Xia, Xintong Li, Aaron Chang, Ryan Rossi, Sachin Kumar, Bodhisattwa Prasad Majumder, Jingbo Shang, Prithviraj Ammanabrolu, Julian McAuley · 2025

Read A Survey on Personalized and Pluralistic Preference Alignment in Large Language Models by Zhouhang Xie, Junda Wu, Yiran Shen, Yu Xia, Xintong Li, Aaron Chang, Ryan Rossi, Sachin Kumar, Bodhisattwa Prasad Majumder, Jingbo Shang, Prithviraj Ammanabrolu, Julian McAuley (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-07070.txt.

Description

Zhouhang Xie and ten co-authors survey, in a 2025 preprint, how language models can be tuned to match each user's own tastes. Standard alignment trains a model toward one shared set of preferences. Real users differ, though, in values, style, and needs. Personalized alignment tries to fit the model to the person. The survey organizes a fast-growing pile of work on that problem. It builds a taxonomy of the methods on offer.

Methods split into three families: those that act during training, those that act at answer time, and those that first build a model of the user. The authors weigh what each family does well and where it strains. They also review how the field tests these systems and what benchmarks exist. Open problems include learning tastes from little data and respecting many cultures at once. The paper offers a shared map for researchers entering the area.

Catalog record

CREATORZhouhang Xie, Junda Wu, Yiran Shen, Yu Xia, Xintong Li, Aaron Chang, Ryan Rossi, Sachin Kumar, Bodhisattwa Prasad Majumder, Jingbo Shang, Prithviraj Ammanabrolu, Julian McAuley
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE33 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2504-07070

Browse the full library · About the Tower of Babel