Language models struggle with compartmentalization
Thomas Vincent Howe, David Wingate · 2026
Read Language models struggle with compartmentalization by Thomas Vincent Howe, David Wingate (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-19284.txt.
Introduction
Thomas Howe and David Wingate show that language models often wall off the same idea when it appears in different forms. A fact in English and Swahili, or a function in Python and Haskell, may be learned twice over instead of once. That duplication wastes capacity and weakens learning as more versions pile up, and added parallel data may not fix it. In small models, early multilingual learning stays almost fully split, and every fix they test flips in effectiveness once the count of versions passes a threshold.
Catalog record
| CREATOR | Thomas Vincent Howe, David Wingate |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 46336 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.19284
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Language models struggle with compartmentalization Thanks: Code: https://github.com/vinhowe/compartmentalization . Eval data: https://doi.org/10.5281/zenodo.20171021 Thomas Vincent Howe, David Wingate Abstract In the training data used by large language models (LLMs), the same latent concept is often presented in multiple distinct ways: the same facts appear in English and Swahili; many functions can be expressed in both Python and Haskell; we can express propositions in both formal and natural language. We show that LLMs can exhibit compartmentalization , where they fail to identify and share statistical strength between distinct presentations of unified concepts. In the worst case, LLMs simply learn parallel internal representations of each presentation of the concept, saturating model capacity with redundancies and decreasing sample efficiency with the number of such presentations. We also demonstrate that synthetic parallel data can fail to improve this despite being easily learned itself. Under this framework, we find that, for small models, early multilingual learning is nearly entirely compartmentalized. Finally, all interventions that we study exhibit a phase transition in which their effectiveness depends on the number of distinct presentations, suggesting that the language modeling objective may only inconsistently unify representations. 1 Introduction Language models learn from data abundant with distinct presentations of unified concepts. The same facts appear in English and Swahili; many functions can be expressed in both Python and Haskell; we can express propositions in both formal and natural language. We would like the model to recognize these as related, and much of the success of large language models suggests that they often do, at least with enough scale. But language models at frontier scale fail in surprising ways. • In the multilingual setting, 6 show that, at 7B scale, language models know more than three times as many facts in any language than the multilingual average, and that task performance across languages does not imply that languages “learn from each other,” i.e., share representations for similar tasks. • Relatedly, 4 demonstrate that non-thinking frontier language models consistently fail to recall Wikipedia facts that exist only in a single language in other languages. • Within a single language, 3 find that frontier language models can correctly recite many facts in a pretraining-like Wikipedia context, but not in a QA-like context. In these cases, encoding succeeds but retrieval fails—the model does not functionally “know” these facts. • In a similar vein, 2 introduce the Reversal Curse, one of the more well-known and well-replicated frontier failures, wherein LLMs fail to generalize from factual relations of the form "A is B" to the implied but undemonstrated "B is A". (a) A demonstration of slowdown. (b) Sample-efficiency cost. (c) Capacity cost. Figure 1 : Compartmentalization costs sample efficiency and capacity. 1(a) How slowdown is calculated: at any val-loss target L achieved by some checkpoint of the base c{=}1 model (here L{=}4.5 ), find the iterations where c{=}1 and c{=}N each cross L and take their ratio. We linearly interpolate to find the intersection point for the compartmentalized model. The example reads 5.0\times for c{=}8 at L{=}4.5 on the 41.9M model. 1(b) Slowdown of each c -compartment run vs. the c{=}1 baseline on the 41.9M model, plotted against the matched val-loss target. Slowdown grows with c across the trajectory. 1(c) Final val loss vs. c , across six model scales (1.1M to 0.98B parameters). Final val loss also grows with c . In understanding this class of failures, we take inspiration from the field of machine translation, which has long held the interlingua as a target: a shared semantic space that multiple languages share. 7 showed that shared encoders can, sometimes, discover representations that generalize across languages they were never jointly trained on. Why then is there so much evidence that frontier models often learn language-specific—and broadly context-specific—representations? We propose a perspective on this type of failure based on redundancy in representation learning. Specifically, we define compartmentalization as a phenomenon in which a model treats c presentations of a latent data generation process as partially or completely distinct, requiring as many as c distinct representations of that structure. The worst case of compartmentalization is redundancy in the strong sense: the same structure, learned independently, stored as separate representations, with no amortization between them. We suggest that this has at least two consequences: • Sample inefficiency. If every latently-similar presentation is treated as a separate prediction task, each presentation benefits only from its own share of the data. In the worst case, where one such presentation accounts for only \varepsilon of the data, it could take 1/\varepsilon as many data samples to learn a similar amount of structure under that presentation, assuming no capacity constraints. • Capacity competition. Redundant representations also consume finite representational capacity. Without committing to a particular notion of capacity, we hypothesize that perfect c -fold redundancy prevents amortizing across presentations, and therefore requires more capacity to represent than the same data under a single presentation the model represents more cohesively. Even with more data, in the capacity-constrained regime, a compartmentalized model might saturate sooner—that is, when evaluated on held-out examples in our worst-case presentation, a model trained on 1/\varepsilon as many examples could still plateau above a model the same size trained only on that presentation. Our contributions within this framework are six-fold. 2 Contributions • We define a data augmentation we show allows us to induce compartmentalization in language models, and demonstrate that within this construction language models pay sample-efficiency and capacity costs that scale with c across six model sizes from 1.1M to \sim 1B parameters. • We provide existence proofs that these costs are not fundamental, with two separate methods that each provide unified solutions SGD on its own does not find: models that can predict within compartments without a compartmentalization tax. • We find that paired “translation” data, despite being easily learned by the model, fails to reduce compartmentalization until a c -dependent phase transition that we replicate at 1B parameter scale—we demonstrate that this is accelerated to lower c by weight decay. • We show that an auxiliary contrastive representation alignment objective exhibits a similar c -dependence; for c{=}2 it provides no improvement in 1M steps, but improvement grows with c . • We propose an operational measure of compartmentalization for models trained on existing datasets with multiple distinct presentations and apply it to multilingual training. • With models trained on a biography/Q&A dataset, we observe destructive capacity competition between multiple formats encoding the same data, with a compartmentalized model as a baseline. 3 A worst-case model of compartmentalization (a) English val loss over training. (b) Slowdown vs. the single-compartment baseline. Figure 2 : The amount of structure in each compartment determines the capacity cost of compartmentalization. All runs use a 14.7M base model with c{=}2 ; we compare four choices for the second compartment’s data—English (a homogeneous control), Russian, unigram-frequency-sampled noise, and uniform-random-token noise—against the single-compartment ( c{=}1 ) baseline. 2(a) English-side validation loss for each condition. 2(b) Slowdown of each c{=}2 run vs. the c{=}1 baseline at matched English val loss. When both compartments contain language data, the English sample efficiency plateaus vs. baseline, but with noise, English data matches baseline sample efficiency. To systematically explore compartmentalization, we first contribute a simple method of creating c distinct