Beyond the Mean: Three-Axis Fidelity for Aligning LLM-Based Survey Simulators from Small Pilot Data
Eun Cheol Choi, Youngrae Kim, Prabhu Pugalenthi, Hong-En Chen, Bo-Ruei Huang · 2026
Read Beyond the Mean: Three-Axis Fidelity for Aligning LLM-Based Survey Simulators from Small Pilot Data by Eun Cheol Choi, Youngrae Kim, Prabhu Pugalenthi, Hong-En Chen, Bo-Ruei Huang (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2606-28963.txt.
Description
Choi, Kim, Pugalenthi, Chen and Huang ask whether a language model, given a small pilot survey, can reproduce the statistics of a whole population. Models used as survey simulators show systematic biases: answers skew, variance is badly calibrated, and links between predictors and outcomes shrink. The team judge recovery on three axes: structural, marginal and individual fidelity, meaning the shape of relationships, the overall distributions and person-level accuracy.
A COVID-19 misinformation survey serves as the case study. Three families of methods compete: prompting the model, fixing its outputs afterwards, and fine-tuning it on the pilot data. Fine-tuning on small pilot samples, they find, offers the most balanced recovery across all three axes. But the scores varied across subsamples. That variation, they warn, can threaten pluralistic alignment, the goal of representing a population's real diversity rather than flattening it toward the average.
Catalog record
| CREATOR | Eun Cheol Choi, Youngrae Kim, Prabhu Pugalenthi, Hong-En Chen, Bo-Ruei Huang |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 37 KB |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2606.28963