Convergence Without Understanding: When Language Models Agree on Representations but Disagree on Reasoning

Muhammad Usama [email protected], Dong Eui Chang [email protected] · 2026

Read Convergence Without Understanding: When Language Models Agree on Representations but Disagree on Reasoning by Muhammad Usama [email protected], Dong Eui Chang [email protected] (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-23315.txt.

Introduction

Usama and Chang test shared likeness across 16 language models from 8 families on 800 thinking problems spanning math, science, common sense, and truth. Their study reveals three splits. First, a hardness flip: models converge more on problems they fail as a group than on those they solve. Second, a making gap: pre-choice codes align while post-choice codes diverge. Third, surface rightness: shared data is read across models but exerts minimal causal pull.

Catalog record

CREATORMuhammad Usama [email protected], Dong Eui Chang [email protected]
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE43161
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.23315

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Convergence Without Understanding: When Language Models Agree on Representations but Disagree on Reasoning Muhammad Usama [email protected], Dong Eui Chang [email protected] Abstract Large language models trained under diverse objectives and architectures have been shown to develop increasingly similar internal representations, an observation formalized as the Platonic Representation Hypothesis. Whether this representational convergence extends to the reasoning processes that operate over shared representations remains untested. We evaluate representational similarity across 16 language models from 8 families (1.5B to 72B parameters) on 800 reasoning problems spanning mathematics, science, commonsense, and truthfulness, stratifying by problem difficulty, computational stage, and causal relevance. Our analysis reveals three dissociations: a difficulty inversion , where models converge more on problems they collectively fail (Centered Kernel Alignment [CKA] = 0.897) than on those they solve (CKA = 0.830); a generation gap , where pre-decision representations align (CKA = 0.875) while post-decision representations diverge (CKA = 0.274); and epiphenomenal correctness , where shared information is decodable across models (66% transfer accuracy) but exerts minimal causal influence on predictions (1.5% to 5.5% flip rate across ablation protocols). These results indicate that representational convergence in language models reflects shared input processing constraints rather than shared reasoning strategies, with direct implications for ensemble design, interpretability transfer, and evaluations of model similarity. Code is available at https://github.com/Usama1002/convergence-without-understanding . Muhammad Usama and Dong Eui Chang † † heading: XX 2026 1- XX-XXXX † † shortheadings: Convergence Without Understanding / Usama and Chang † † firstpage: 1 keywords representational similarity, language models, Platonic Representation Hypothesis, mechanistic interpretability, centered kernel alignment, reasoning, causal ablation 1 Introduction Neural networks trained with different architectures, objectives, and datasets develop increasingly similar internal representations as they scale ( 20 ) . This empirical regularity, formalized as the Platonic Representation Hypothesis (PRH), posits that diverse models converge toward a shared statistical model of reality that reflects the structure of the data-generating process rather than the idiosyncrasies of individual training regimes. The next-token prediction objective promotes linearly accessible concept structure, providing theoretical grounding for this convergence ( 22 ) , and cross-modal extensions have demonstrated that representations across vision, language, and audio modalities converge toward common geometric structures ( 17 ; 23 ) . Dataset and task overlap are causal drivers of this phenomenon ( 27 ) , and linear transformations can bridge representations across LLMs for behavioral transfer ( 19 ) . The implication is striking: if true, the PRH would provide a principled basis for transfer learning, model interpretability, and claims about what neural networks learn. But what, precisely, does this convergence mean? The existing evidence rests almost entirely on aggregate similarity measures computed across entire datasets without conditioning on what models do with those representations. Centered Kernel Alignment (CKA) ( 25 ) is the current standard, with SVCCA ( 33 ) , CCA-based variants ( 31 ) , and mutual nearest neighbor methods providing complementary perspectives ( 24 ; 12 ) . Yet each metric carries known limitations: CKA is sensitive to transformations that do not change functional behavior ( 11 ) , can be driven to near-maximum values under certain sample-feature ratios ( 32 ) , and input statistics can inflate scores without deconfounding ( 10 ) . More fundamentally, none of these studies condition similarity on the computational processes that transform representations into predictions. 6 proved analytically that representational and functional similarity can be entirely decoupled. We refer to this as the problem of convergence without understanding, and we test it empirically. We directly test whether representational convergence implies reasoning convergence by studying 16 language models spanning 8 architectural families (1.5B to 72B parameters) on 800 reasoning problems drawn from GSM8K ( 9 ) , ARC-Challenge ( 8 ) , TruthfulQA ( 28 ) , and HellaSwag ( 41 ) . Beyond computing aggregate similarity scores per benchmark, we stratify along three axes that prior work has not examined jointly: problem difficulty (fraction of models solving each problem), computational stage (pre-decision versus post-decision layers), and causal relevance (whether shared representations drive predictions). This stratified analysis reveals three findings that collectively challenge the standard interpretation of representational convergence. First, we observe a difficulty inversion : models are more representationally similar on problems they collectively fail (CKA = 0.897) than on those they collectively solve (CKA = 0.830), a pattern confirmed by mutual nearest neighbors, replicated across three of four reasoning domains, and persistent at the 70B scale (Section 3.1 ). This is unexpected under the PRH, which predicts maximal convergence where models successfully represent reality, particularly given that language models develop rich internal representations including world models, spatial structure, and structured beliefs ( 26 ; 18 ; 43 ) . We trace the mechanism to attention entropy: hard problems produce diffuse attention that homogenizes representations across architectures ( 42 ; 14 ) . Second, we identify a generation gap : pre-decision representations converge (CKA = 0.875) while post-decision representations diverge (CKA = 0.274), indicating that convergence is primarily a property of input encoding rather than output computation (Section 3.2 ). This extends the encoding-generation distinction identified in single-model factual recall studies ( 16 ) to the cross-model setting, and explains why model stitching ( 4 ) and function vector transfer ( 37 ) succeed for early layers but degrade for late layers. Third, we demonstrate epiphenomenal correctness : correctness information transfers across models (66% probe accuracy) but does not causally influence predictions (1.5% to 5.5% full-subspace flip rate across ablation protocols), establishing that shared representations encode task-relevant information without deploying it (Section 3.3 ). This extends the correlational-causal gap identified within individual models by the probing literature ( 5 ; 38 ; 29 ) to the cross-model setting: cross-model transfer probes ( 19 ) confirm shared encoding, yet our causal ablation shows it is not deployed. Two additional findings calibrate scope: randomly initialized models exhibit higher CKA than trained models, indicating that much of the observed convergence is architectural rather than learned ( 15 ) , and the inversion replicates more strongly in base (non-instruction-tuned) models, ruling out alignment training as its source ( 36 ) . These findings have direct practical consequences for ensemble design, interpretability transfer, and evaluations of model similarity. 2 Methodology 2.1 Models and Tasks We study 16 instruction-tuned language models spanning 8 families (1.5B to 72B parameters); the full list is in Table 1 . These models differ in training data, tokenizer vocabulary, positional encoding, and attention mechanism, while the diversity of alignment procedures (RLHF, RLAIF, supervised fine-tuning) preserves training recipe variation. To test scale invariance, we additionally evaluate LLaMA-3.1-70B and Qwen-2.5-72B using two A100 80GB GPUs. Model accuracies range from 24.5% to 70.9% for the 14-model core cohort and reach 80.1% for the 72B models. Table 1 : Models evaluated in this study. Accuracy is the fraction of 800 problems answered correctly across all four domains. Families span 8 distinct architectural lineages with diverse training recipes. Model Family Params Layers Acc. (%) Qwen-2.5-1.5B ( 40 ) Qwen 1.5B 28 55.0 SmolLM2-1.7B ( 3 ) SmolLM 1.7B 24 41.6 Gemma-1-2B ( 30 ) Gemma 2.0B 18 24.5 Gemma-2-2B ( 34 ) Gemma 2.6B 26 64.8 Qwen-2.5-3B ( 40 ) Qwen 3.0B 36 56.4 LLaMA-3.2-3B ( 13 ) LLaMA 3.2B 28 63.6 Phi-3.5-Mini ( 1

Read the complete text