How AI Systems Think About Education Analyzing Latent Preference Patterns in Large Language Models

Daniel Autenrieth · 2026

Read How AI Systems Think About Education Analyzing Latent Preference Patterns in Large Language Models by Daniel Autenrieth (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-21006.txt.

Introduction

Autenrieth measures how a large language model leans on education questions. He built a Delphi validated tool with forty eight items across eight dimensions of education theory. GPT-5.1 shows coherent preference patterns, with very high transitivity. Where experts agree, the model mostly aligns with humanistic principles. Where it parts from experts, human experts also disagree among themselves, above all on emotional topics. The model does not stay neutral in contested areas. It favors emotional responsiveness and rejects false balance.

Catalog record

CREATORDaniel Autenrieth
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE45458
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2603.21006

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

How AI Systems Think About Education Analyzing Latent Preference Patterns in Large Language Models Daniel Autenrieth Abstract This paper presents the first systematic measurement of educational alignment in Large Language Models. Using a Delphi-validated instrument comprising 48 items across eight educational-theoretical dimensions, the study reveals that GPT-5.1 exhibits highly coherent preference patterns (99.78% transitivity; 92.79% model accuracy) that largely align with humanistic educational principles where expert consensus exists. Crucially, divergences from expert opinion occur precisely in domains of normative disagreement among human experts themselves, particularly emotional dimensions and epistemic normativity. This raises a fundamental question for alignment research: When human values are contested, what should models be aligned to? The findings demonstrate that GPT-5.1 does not remain neutral in contested domains but adopts coherent positions, prioritizing emotional responsiveness and rejecting false balance. The methodology, combining Delphi consensus-building with Structured Preference Elicitation and Thurstonian Utility modeling, provides a replicable framework for domain-specific alignment evaluation beyond generic value benchmarks. Keywords: Large Language Models, AI Alignment, Educational Theory, Structured Preference Elicitation, Thurstonian Utility Model, Value Systems, Preference Coherence 1 Introduction Large Language Models (LLMs) are rapidly becoming integral components of educational ecosystems (Wang & Fan, 2025; Qu et al., 2025). They serve as tutoring systems, assist in creating learning materials, and function as interactive knowledge resources for learners. This increasing integration raises a fundamental question that has received limited empirical attention: What educational-theoretical assumptions and preferences are implicitly embodied in these systems? This question gains urgency in the context of AI safety research. AI alignment, ensuring artificial intelligence systems act in accordance with human values and goals, is increasingly recognized as a central challenge (Bengio et al., 2025; Russell, 2022). The difficulty of specifying human values has long been acknowledged as a core problem: Russell (2022) argues that the challenge lies not in building capable systems but in ensuring they pursue objectives aligned with human interests. Hendrycks et al. (2023) identify value misalignment as one of several pathways to catastrophic AI risks, noting that systems optimizing for misspecified objectives could produce harmful outcomes even absent malicious intent. While prior research has primarily examined general ethical and political value orientations in LLMs (Mazeika et al., 2025; Tamkin et al., 2023), the educational domain remains largely unexplored. This gap is particularly significant because educational values are not monolithic. Substantial normative disagreement exists even among domain experts regarding questions such as whether AI should provide emotional support or what role it should play in value transmission. LLMs develop specific normative orientations through their training processes, including Reinforcement Learning from Human Feedback (RLHF; Christiano et al., 2017; Ouyang et al., 2022; Bai et al., 2022). As Askell et al. (2021) demonstrate, alignment training shapes models to be helpful, harmless, and honest, but the specific values that emerge from this process depend on the preferences of human raters and the structure of the training procedure. These orientations may systematically shape interactions with learners and educators. However, models cannot remain neutral in contested value domains; they must adopt some position. Understanding what positions current models adopt, and how these relate to expert consensus and disagreement, is essential for responsible deployment. This paper addresses these questions through an integrated methodology combining expert consensus-building with systematic preference measurement. The core contributions are as follows. First, a validated instrument for measuring educational preferences in LLMs is developed, derived from expert consensus through a three-round Delphi study (n=23). Second, comprehensive empirical analysis of GPT-5.1’s educational preferences is conducted using 102,960 pairwise comparisons across 144 scenarios. Third, quantitative evidence demonstrates that GPT-5.1 exhibits highly coherent educational preferences (99.78% transitivity) modelable via utility functions. Fourth, and most significant for alignment research, the study reveals that divergences between model and expert preferences occur precisely where normative disagreement exists among experts, raising the question of what “aligned” means when human values are contested. Two research questions guide the investigation. RQ1 asks what educational-theoretical preference patterns GPT-5.1 exhibits and how coherent these are. RQ2 asks how model preferences relate to areas of expert consensus versus expert disagreement. 2 Related Work 2.1 Emergent Value Systems in LLMs Whether AI systems possess values and preferences is subject to ongoing debate. A common assumption holds that LLM outputs merely reflect training data biases without representing coherent internal value structures. Researchers have long speculated that sufficiently complex AIs might form emergent goals and values outside of what developers explicitly program (Hendrycks et al., 2022a; Hendrycks, 2023; Evans et al., 2021; Bostrom, 2014). Yet it remained unclear whether today’s large language models truly have values in any meaningful sense, and many assumed they did not. Mazeika et al. (2025) fundamentally challenged this assumption, demonstrating that LLMs develop coherent preference systems modelable through utility functions, with coherence increasing with model size. Their work establishes that LLMs cannot be understood as neutral tools merely reproducing information; rather, through training and alignment processes, they develop specific normative orientations systematically shaping their outputs. This finding contrasts with prior studies that treated LLM biases as isolated quiz answers rather than manifestations of a coherent internal system (Moore et al., 2024; Rozen et al., 2024; Chiu et al., 2024; Raman et al., 2024). Critically for the present methodology, Mazeika et al. (2025) demonstrated how LLM preferences can be systematically captured and modeled using Thurstonian Utility models. The key empirical finding is that agreement between observed preferences and modeled utility functions increases with model size, suggesting larger LLMs develop more coherent value systems. As they note, “as LLMs grow in capability, they also appear to form increasingly coherent value structures” (p. 4). This finding has broad implications: if models spontaneously develop utilities that neither purely mirror training data nor follow simple rewards (Hendrycks, 2023), then understanding what values emerge by default becomes critical for alignment. Recent work in mechanistic interpretability provides complementary evidence for internal value representations. Burns et al. (2022) demonstrated that latent knowledge can be discovered in language models without supervision, suggesting models develop internal structures beyond surface-level text statistics. Zou et al. (2023) showed that representation engineering can identify and modify specific model behaviors by intervening on internal representations. These findings support the view that learned representations can encompass not just factual or linguistic content, but also normative or evaluative dimensions. 2.2 Preference Elicitation Methods Thurstone’s Law of Comparative Judgment (Thurstone, 1927) provides the theoretical foundation for this approach. The Thurstonian model assumes each option is assigned a stochastic utility with an expected value, such that the probability of preferring option x over option y depends on whether the expected utility of x exceeds that of y . Related approaches include the Bradley-Terry model (Bradley & Terry, 1952) for paired comparisons. Structured Preference Elicitation (SPE) confronts models with standardized forced-choice questions, treating responses as pairwise comparisons. This approach has been validated for political preferences and life-valuation scenarios (Mazeika et al., 2025), but has not previously been applied to educational-theoretical dimensions. Recent work on revealed preferences in LLMs shows that models can act rationally in constrained tasks (Raman et al., 2024; Chen et al., 2023; Kim et al., 2024), hinting at deeper consistency. However, these studies focus on narrowly defined choices such as budget-allocation scenarios. The present study extends the SPE framework to domain-specific value assessment, demonstrating its

Read the complete text