Systematic Generalization in Language Models Scales with Information Entropy
Sondre Wold, Lucas Georges Gabriel Charpentier, Étienne Simon · 2025
Read Systematic Generalization in Language Models Scales with Information Entropy by Sondre Wold, Lucas Georges Gabriel Charpentier, Étienne Simon (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-13089.txt.
Introduction
Sondre Wold, Lucas Georges Gabriel Charpentier, and Étienne Simon ask what makes a systematic generalization problem hard for language models. Models often stumble when known pieces appear in new combinations, yet no ruler measured that difficulty. The authors propose one: the entropy of how parts are spread in training data.
They build a formal way to compute this entropy for sequence tasks. Across popular model designs, scores rose and fell with entropy in a scaling pattern. High entropy success came even without built-in biases. Low entropy remains hard for everyone, which the team offers as the real target for progress toward sturdy compositional skill.
Catalog record
| CREATOR | Sondre Wold, Lucas Georges Gabriel Charpentier, Étienne Simon |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 43 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-13089
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Systematic Generalization in Language Models Scales with Information Entropy Sondre Wold, Lucas Georges Gabriel Charpentier, Étienne Simon Abstract Systematic generalization remains challenging for current language models, which are known to be both sensitive to semantically similar permutations of the input and to struggle with known concepts presented in novel contexts. Although benchmarks exist for assessing compositional behavior, it is unclear how to measure the difficulty of a systematic generalization problem. In this work, we show how one aspect of systematic generalization can be described by the entropy of the distribution of component parts in the training data. We formalize a framework for measuring entropy in a sequence-to-sequence task and find that the performance of popular model architectures scales with the entropy. Our work connects systematic generalization to information efficiency, and our results indicate that success at high entropy can be achieved even without built-in priors, and that success at low entropy can serve as a target for assessing progress towards robust systematic generalization. 1 Introduction Human language is characterized by its combinatorial properties in syntax and semantics ( Hadley 1994 ) . For example, when someone knows what it means for Person A to see Person B, they can readily generalize to what it means for Person B to see Person A, without having been exposed to examples where Person B is used in the subject position ( Fodor and Pylyshyn 1988 ) . This type of bidirectional systematic generalization has long been considered a key feature of human language understanding ( Chomsky 1957 ) , and is closely related to the compositionality of natural language semantics, a contentious but widely studied topic in both formal semantics ( van Bethem et al. 1991 ) and NLP ( McCurdy et al. 2024 ) . 0 1 2 3 0 0.2 0.4 0.6 0.8 1 Entropy Accuracy CNN RNN Trans. Figure 1: Systematic generalization increases with the entropy of the training data for popular sequence-to-sequence models, even without built-in priors. Early critics of connectionism, such as Fodor and Pylyshyn 1988 , argued that systematic generalization was incompatible with continuous representations. More than thirty years since this critique was initially presented, researchers still agree that achieving robust compositional behavior in neural networks remains an open problem, as shown by a recent survey by McCurdy et al. 2024 . Research continues to demonstrate that current models struggle with systematic generalization ( Lake and Baroni 2018 ; Hupkes et al. 2020 ; Dziri et al. 2024 ; Wold et al. 2024 ; Mirzadeh et al. 2024 ) . Some of the limitations of current models can be alleviated by implementing compositional priors, either in the architecture ( Gordon et al. 2019 ) or through the training procedure ( Lake and Baroni 2023 ) , but it remains unclear whether architectural constraints cause the limitations or whether they arise due to distributional properties of the data used for training. v_{1} v_{2} v_{3} v_{4} 0% 25% 50% 75% 100% verbs v_{1} v_{2} v_{3} v_{4} 0% 25% 50% 75% 100% verbs v_{1} v_{2} v_{3} v_{4} 0% 25% 50% 75% 100% verbs v_{1} v_{2} v_{3} v_{4} 0% 25% 50% 75% 100% verbs \mathrm{H}=0 \mathrm{H}=1 \mathrm{H}=1.5 \mathrm{H}=2 Entropy v_{1} v_{2} v_{3} v_{4} 0% 25% 50% 75% 100% verbs v_{1} v_{2} v_{3} v_{4} 0% 25% 50% 75% 100% verbs v_{1} v_{2} v_{3} v_{4} 0% 25% 50% 75% 100% verbs v_{1} v_{2} v_{3} v_{4} 0% 25% 50% 75% 100% verbs \mathrm{H}=0 \mathrm{H}=1 \mathrm{H}=1.58 \mathrm{H}=2 Systematic generalization Figure 2: A schematic overview of the two approaches to increasing the entropy level. Top: Vertical scaling by distribution mixing. Bottom: Horizontal scaling by incrementing the support of the distribution. In this paper, we ask the following question: Can neural networks display systematic out-of-distribution generalization without any built-in priors? To answer this, we take a data-centric approach. Specifically, we show how the degree of systematic generalization of common architectures relates to the information entropy of the distribution of component parts found in the training data. Contrary to previous work, we find that popular sequence-to-sequence models used in NLP are capable of systematic generalization, even without built-in architectural priors. However, this is only the case when the training dataset contains high entropy. On a series of sequence-to-sequence datasets based on a modified version of the SCAN ( Lake and Baroni 2018 ) grammar, we show that model performance scales with the entropy of the training dataset, as illustrated in Figure 1 , thereby creating a link between systematic generalization and information efficiency. In summary, our contributions are: (i) We develop and formalize a framework for studying systematic generalization based on information entropy; (ii) We experiment with two different approaches to increasing entropy, and we compare the performance of four different model architectures for these two settings; and (iii) we show that neural networks achieve systematic out-of-distribution generalization when the entropy of the training distribution is high, independently of the number of unique samples seen during training. 2 Background Systematic generalization is typically described as a robust mode of composition, where a system is able to combine parts of its training data in a way that enables generalization to novel combinations. As such, systematic generalization relates to compositionality, a topic which has both a long tradition and multiple formal definitions, most prominently in formal semantics ( van Bethem et al. 1991 ; Pagin and Westersthl 2010 ) . These definitions, however, often prove too restrictive when working with natural languages. As a result, researchers frequently introduce problem-specific relaxations, leading to ambiguity over time regarding the exact meaning of terms such as systematic generalization and compositionality. Despite there being some ambiguity regarding the formal definitions, recent work shows that there is high agreement among researchers on the following, more informal definition of what constitutes compositional behavior: Definition 1 (CB) “When a model receives an input I that humans conceive as composed of component parts, if the model produces correct outputs for those parts (in isolation or in other combinations), then it will also produce a correct output for I ( McCurdy et al. 2024 , p. 9324) .” This definition, however, does not consider the conditions from which a system learns to produce these outputs. Consequently, compositional behavior differs from compositional generalization . While the former can be assessed through normal benchmarks, the latter requires information about the distribution the model generalizes from. Although generalization is a loosely defined term in machine learning research, it generally refers to how well a model performs on a held-out dataset after being trained on data that has some distributional relationship with the held-out data ( Hupkes et al. 2023 ) . When training and test distributions are similar, we refer to this as in-distribution , and when they differ, we typically refer to this as out-of-distribution . Naturally, the classification of a generalization problem as in-distribution or out-of-distribution depends on which aspects of the distribution are considered. In this work, we focus on systematic generalization as an out-of-distribution problem, where the differences in the distribution of component parts and their combinations between the training and test data are the relevant aspects. 2.1 Systematic Generalization Following earlier works, we will refer to the systematic generalization abilities of a model as the systematicity of