Alignment Makes Language Models Normative, Not Descriptive
Eilam Shapira, Moshe Tennenholtz, Roi Reichart · 2026
Read Alignment Makes Language Models Normative, Not Descriptive by Eilam Shapira, Moshe Tennenholtz, Roi Reichart (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-17218.txt.
Introduction
Shapira, Tennenholtz, and Reichart show that aligned language models predict what people should do, not what they do. They compare 120 base and aligned model pairs on over 10,000 real human decisions in strategic games. In multi round games like bargaining and negotiation, base models beat aligned ones at predicting human choices by nearly ten to one. The pattern flips in one shot textbook games and simple lottery choices, where aligned models win. The authors read this as a normative bias from alignment.
Catalog record
| CREATOR | Eilam Shapira, Moshe Tennenholtz, Roi Reichart |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 72387 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.17218
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Alignment Makes Language Models Normative, Not Descriptive Eilam Shapira, Moshe Tennenholtz, Roi Reichart Abstract Post-training alignment optimizes language models to match human preference signals, but this objective is not equivalent to modeling observed human behavior. We compare 120 base–aligned model pairs on more than 10,000 real human decisions in multi-round strategic games—bargaining, persuasion, negotiation, and repeated matrix games. In these settings, base models outperform their aligned counterparts in predicting human choices by nearly 10:1, robustly across model families, prompt formulations, and game configurations. This pattern reverses, however, in settings where human behavior is more likely to follow normative predictions: aligned models dominate on one-shot textbook games across all 12 types tested and on non-strategic lottery choices — and even within the multi-round games themselves, at round one, before interaction history develops. This boundary-condition pattern suggests that alignment induces a normative bias: it improves prediction when human behavior is relatively well captured by normative solutions, but hurts prediction in multi-round strategic settings, where behavior is shaped by descriptive dynamics such as reciprocity, retaliation, and history-dependent adaptation. These results reveal a fundamental trade-off between optimizing models for human use and using them as proxies for human behavior. Figure 1: Pearson correlations of base models and human decisions (x-axis) vs. aligned models and human decisions (y-axis) across four game families. Each point is a same-provider pair evaluated in its native format (standard prompt for base, chat template for aligned). Points below the diagonal indicate base advantage. The shaded region marks pairs where both models correlate below 0.3 with human behavior. Base models win 75:4 in bargaining, 32:4 in persuasion, 25:1 in negotiation, and 81:13 in matrix games, for an overall ratio of 9.7:1 (213 vs. 22, p<10^{-40} ). 1 Introduction Large language models (LLMs) are increasingly used as proxies for human behavior ( 11 ; 1 ; 5 ; 3 ; 28 ; 13 ; 34 ) . They replicate classic experimental findings from psychology and economics, approximate subgroup opinion distributions when conditioned on demographic backstories, and predict survey experiment outcomes. The approach extends to strategic settings: LLMs can predict human decisions in language-based persuasion games, outperforming models trained on human data alone ( 30 ) , and capture cooperation patterns in repeated social dilemmas ( 2 ; 22 ) . Yet nearly all of this work uses aligned models, treating alignment as either neutral or beneficial for behavioral prediction. This assumption deserves scrutiny. Alignment via RLHF ( 25 ) or DPO ( 26 ) optimizes models for responses that human evaluators approve of —cooperative, fair, and socially appropriate. But human behavior in strategic settings is often none of these: people bluff, retaliate, and deviate from approved patterns ( 9 ; 4 ) . If alignment narrows the model’s behavioral distribution toward such responses ( 16 ; 8 ; 12 ) , it creates a normative bias —the model learns to predict behavior that people endorse rather than behavior they exhibit . The distinction between normative theories (how people should act) and descriptive accounts (how people actually act) is foundational in the social and behavioral sciences ( 6 ) . This predicts that aligned models should predict human behavior well in settings where that behavior is relatively simple and well-described by normative theory, but poorly where behavior is complex and shaped by interaction history. Multi-round strategic games—where decisions depend on accumulated experience with a specific opponent—provide a natural test case for the descriptive end: behavior there is driven by reciprocity, retaliation, and reputation dynamics. One-shot decisions over well-studied game structures or simple lotteries provide a contrasting case where normative predictions may be more accurate. We test this hypothesis by comparing 120 same-provider base--aligned 1 1 1 Throughout, aligned denotes models that have undergone post-training optimization beyond next-token prediction—typically supervised fine-tuning combined with preference optimization via RLHF or DPO; base denotes the pre-alignment checkpoint. model pairs from 23 families (see Appendix A ) on predicting 10,050 real human decisions across four families of multi-round strategic games: bargaining, persuasion, negotiation, and repeated matrix games (Prisoner’s Dilemma and Battle of the Sexes). By restricting to same-provider pairs, each comparison directly isolates the effect of alignment. Each model is evaluated in its native format: standard text completion for base models, chat-templated input for aligned models. The results are consistent with the hypothesis. In multi-round games, base models outperform their aligned counterparts by a ratio of 9.7:1 (213 vs. 22 wins, p<10^{-40} ), with each game family individually significant ( p<10^{-6} ). The effect holds across all 23 model families, 10 prompt formulations, and all game configuration parameters, and grows with model scale. The hypothesis also predicts where the base advantage should not hold: in simpler settings without multi-round history, normative predictions may suffice, and alignment should help rather than hurt. We test two such boundary conditions—one-shot 2\times 2 matrix games and non-strategic binary lotteries—and find that the advantage reverses in both. Aligned models win 4.1:1 on one-shot games ( p<10^{-6} ), consistently across all 12 game types, and 2.2:1 on lotteries ( p<10^{-3} ). In the one-shot games, aligned models’ predictions are closer to Nash equilibrium—which itself correlates with human behavior in these settings ( r=0.62 )—consistent with alignment shifting predictions toward normative patterns. The same reversal appears within multi-round games at round one, before interaction history develops, but disappears as history accumulates. 2 Related Work 2.1 LLMs as Human Behavioral Proxies A growing literature treats LLMs as behavioral models of humans— homo silicus ( 11 ) —capable of replicating experimental findings ( 1 ) , approximating subgroup opinions ( 3 ) , and predicting treatment effects ( 13 ) . Nearly all of this work uses aligned models, implicitly assuming that alignment is neutral for behavioral fidelity. Yet several findings challenge this assumption: RLHF collapses opinion diversity toward specific groups ( 28 ) , instruction tuning introduces cognitive biases absent in base models ( 14 ) , LLMs over-predict normatively rational behavior ( 18 ) , and RLHF-tuned models fail to mirror human response biases ( 35 ) . Most directly, 34 found that aligned models are dramatically worse than base models at zero-shot opinion prediction. These results suggest that alignment distorts behavioral representations—but the evidence comes from opinions and individual judgments. Whether the pattern extends to multi-round strategic interactions , where behavior is shaped by history and reciprocity, remains untested. 2.2 The Alignment Tax Alignment can degrade capabilities beyond helpfulness, a phenomenon termed the “alignment tax.” Base models outperform aligned variants on reasoning benchmarks ( 23 ) , and calibration deteriorates across the tuning pipeline ( 15 ; 37 ) . More fundamentally, alignment narrows the model’s output distribution: RLHF significantly reduces output diversity ( 16 ) , and the standard KL-regularized RL framework can only specify unimodal targets, making diversity collapse a built-in feature rather than an implementation failure ( 12 ; 17 ; 36 ) . These results establish that alignment narrows distributions and why , but measure the cost in generation quality and benchmark scores—not in behavioral prediction fidelity. Whether distributional narrowing degrades a model’s ability to predict the full range of human strategic behavior has not been tested directly. 2.3 LLMs in Strategic Games Prior work studies how LLMs play games