Beyond Sociodemographic Prompting: Using Supervision to Align LLMs with Human Response Distributions

Gauri Kambhatla, Sanjana Gautam, Angela Zhang, Alex Liu · 2025

Read Beyond Sociodemographic Prompting: Using Supervision to Align LLMs with Human Response Distributions by Gauri Kambhatla, Sanjana Gautam, Angela Zhang, Alex Liu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-00439.txt.

Introduction

Gauri Kambhatla and co-authors test in 2025 if light training helps language models match how real groups answer survey items. Prompt tricks based on age or race give mixed gains at best. The team asks if plain labeled data does a steadier job.

Three data sets cover health, public views, and values and beliefs. Scores check both the mean match and how match shifts group by group. Light training beat prompt tricks in a more steady way across models. The work sets a shared test bed so later teams can gauge group match in the same terms.

Catalog record

CREATORGauri Kambhatla, Sanjana Gautam, Angela Zhang, Alex Liu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE75 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2507-00439

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Beyond Sociodemographic Prompting: Using Supervision to Align LLMs with Human Response Distributions Gauri Kambhatla, Sanjana Gautam, Angela Zhang, Alex Liu Abstract The ability to accurately predict how different population groups would answer subjective questions would have great value. In this work, we show that use of relatively simple supervision can greatly improve language model alignment with diverse population groups, as measured over three datasets spanning various topics. Beyond evaluating average performance, we also report how alignment varies across specific groups. The simplicity and generality of our approach promotes easy adoption, while our broad findings provide useful guidance for when to use or not use our approach in practice. By conducting evaluation over many LLMs and prompting strategies, along with open-sourcing our work, we provide a useful benchmark to stimulate future research 1 1 1 Our data and code will be available at https://github.com/GauriKambhatla/supervised-llm-alignment . 1 Introduction Human surveys elicit valuable public opinion on a wide range of topics, such as proposed legislation, candidates for office (i.e., election polls), marketing or outreach campaigns, commercial products and services, etc. Hayati et al. 2024 ; Santurkar et al. 2023 . Subjective annotation tasks in NLP also seek diverse, human judgments Sap et al. 2022 ; Biester et al. 2022 ; Pei and Jurgens 2023 . Given the frequency, breadth, and importance of obtaining opinion data from diverse populations, there has been great interest in developing effective ways to conduct such surveys accurately and efficiently. Figure 1: Prior work studies using persona, or sociodemographic, prompting to align LLM generations with human responses for subjective questions. In this work, we study eliciting distributions from LLMs and calibrating these distributions to align them with human response distributions. Recent work has sought to simulate human responses using LLMs in many areas, such as in psychology, sociology, and economic studies Aher et al. 2023 , NLP annotation Nasution and Onan 2024 , and large-scale survey creation and testing Rothschild et al. 2024 . More specifically, LLMs are being increasingly used to simulate social or demographic groups Hu and Collier 2024 . This is commonly done with persona or sociodemographic (SD) prompting Hu and Collier 2024 ; Beck et al. 2024 by both technical and non-technical practitioners alike Beck et al. 2024 . Prior work on SD prompting has typically suffered from one or more key shortcomings. First, many studies have assumed a single majority answer for each sociodemographic group in modeling and/or evaluation Hwang et al. 2023 ; Hu and Collier 2024 ; Sun et al. 2025 ; Mukherjee et al. 2024 . The fallacy in such a framing is clear: because members of a given group do not all share the same beliefs, accurate modeling and evaluation must incorporate intra-group disagreement. In addition, prompting for a distribution, rather than a representative response, could potentially make a model less susceptible to caricature and stereotyping Cheng et al. 2023a ; Wang et al. 2024 ; Cheng et al. 2023b , better aligning with the goals of pluralistic alignment Sorensen et al. 2024 . Prior studies modeling distributional beliefs have also tended to evaluate only a single method of extracting an LLM distribution Santurkar et al. 2023 ; Sun et al. 2024 . Other work seeking to usefully replicate ground-truth answer distributions have been agnostic to underlying demographics, i.e., evaluating LLM distributions prompted with specific demographics with ground-truth distributions of multiple other demographics Beck et al. 2024 . In this work, we evaluate SD prompting in a distributional manner across three large survey datasets (both US and global), using a variety of methods to extract distributions from LLMs. We focus on methods that work most broadly, supporting use with both black-box and white-box LLMs. We apply supervised calibration to better align LLM-generated distributions with human response distributions, with the intuition that LLM distributions might be directionally correct, but simply uncalibrated ; e.g., LLM distributions might exaggerate differences between different groups. Concretely, we evaluate and explore distributional opinion alignment by studying the following research questions : (RQ 1) Does SD prompting generate distributions that are more aligned with human opinion? (RQ 2) Can we calibrate LLM-generated distributions to be more aligned with humans through supervised data? Is this consistent across models, datasets, and distribution elicitation methods? (RQ 3) Can generated distributions be more easily aligned with some SD groups over others? What is the effect of calibration for individual groups? (RQ 4) Does calibration work with fewer supervised training examples? And (RQ 5) What are the effects of post-training (methods like SFT and DPO) on LLM-generated distributions? Our experiments evaluate the alignment of LLM-generated distributions to human opinion, across three distribution elicitation methods, and 15 models of varying degrees of openness (open-source, open-model, API-access only), size, pre-training data, and post-training methods. We find that while baseline SD prompting techniques do not lead to consistent improvements in alignment across settings (RQ1), our approach to supervised calibration of LLM-generated distributions improves alignment by 16.3% on average across settings (RQ2). Moreover, as few as 5 gold examples can be used to calibrate distributions (RQ4), though alignment to some demographics is degraded more than others (RQ3). In regard to LLM post-training (RQ5), the effect appears to be dataset dependent, though we find that calibrated alignment is less affected than uncalibrated distributions. We conclude with concrete suggestions to practitioners seeking to predict diverse human responses (e.g., public opinion surveys) via LLMs, offering appropriate caution. 2 Related Work Sociodemographic (SD) Prompting To support safe deployment of models, it is important that modeling and evaluation practices account for social dimensions of diversity Santurkar et al. 2023 that shape our social identities, including sociodemographic factors such as age, gender, and ethnicity Hwang et al. 2023 ; Movva et al. 2024 . SD prompting refers to incorporating demographic variables into input prompts to enhance model alignment with desired outputs Alipour et al. 2024 ; Joshi et al. 2025 . As past work has shown that neither annotator agreement methodologies nor reward mechanisms often produce significant differences in the solution Yang et al. 2024 , prompting methods may support more efficient processes. Some work has reported that SD prompting can be effective in improving model performance Beck et al. 2024 , supporting SD-specific diversification of predictions without need for collecting sensitive annotator SD information Gupta et al. 2023 . Pitfalls of Sociodemographic Prompting Prior studies evaluating sociodemographic prompting have reported mixed results in effectiveness in accurately simulating a social group Beck et al. 2024 ; Santurkar et al. 2023 ; Hu and Collier 2024 . While some works show that sociodemographic prompting can improve performance, results vary greatly by prompt, model, and task Beck et al. 2024 ; Mukherjee et al. 2024 . Other work warns against such methods due to misportrayal, othering, and exoticization of identities Cheng et al. 2023a ; Wang et al. 2024 ; Cheng et al. 2023b . Simply incorporating demographic features into prompts does not always enable LLMs to adopt the perspectives of specific demographic groups Sun et al. 2025 . Furthermore, a vast body of work has established biases in language models that may be perpetuated with sociodemographic prompting Nadeem et al. 2020 ; Gallegos et al. 2024 ; Gupta et al. 2024 . There

Read the complete text