A Survey on Personalized and Pluralistic Preference Alignment in Large Language Models
Zhouhang Xie, Junda Wu, Yiran Shen, Yu Xia, Xintong Li, Aaron Chang, Ryan Rossi, Sachin Kumar, Bodhisattwa Prasad Majumder, Jingbo Shang, Prithviraj Ammanabrolu, Julian McAuley · 2025
Read A Survey on Personalized and Pluralistic Preference Alignment in Large Language Models by Zhouhang Xie, Junda Wu, Yiran Shen, Yu Xia, Xintong Li, Aaron Chang, Ryan Rossi, Sachin Kumar, Bodhisattwa Prasad Majumder, Jingbo Shang, Prithviraj Ammanabrolu, Julian McAuley (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2504-07070.txt.
Description
Zhouhang Xie and ten co-authors survey, in a 2025 preprint, how language models can be tuned to match each user's own tastes. Standard alignment trains a model toward one shared set of preferences. Real users differ, though, in values, style, and needs. Personalized alignment tries to fit the model to the person. The survey organizes a fast-growing pile of work on that problem. It builds a taxonomy of the methods on offer.
Methods split into three families: those that act during training, those that act at answer time, and those that first build a model of the user. The authors weigh what each family does well and where it strains. They also review how the field tests these systems and what benchmarks exist. Open problems include learning tastes from little data and respecting many cultures at once. The paper offers a shared map for researchers entering the area.
Catalog record
| CREATOR | Zhouhang Xie, Junda Wu, Yiran Shen, Yu Xia, Xintong Li, Aaron Chang, Ryan Rossi, Sachin Kumar, Bodhisattwa Prasad Majumder, Jingbo Shang, Prithviraj Ammanabrolu, Julian McAuley |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 33 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2504-07070