How Do Transformers Learn to Associate Tokens: Gradient Leading Terms Bring Mechanistic Interpretability
Shawn Im, Changdae Oh, Zhen Fang, Sharon Li · 2026
Read How Do Transformers Learn to Associate Tokens: Gradient Leading Terms Bring Mechanistic Interpretability by Shawn Im, Changdae Oh, Zhen Fang, Sharon Li (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-19208.txt.
Introduction
Shawn Im and three co-authors open up how transformers learn that certain words belong together. Links like the one between bird and flew let models generalize instead of merely memorizing text. Knowing how such links form is key to a mechanical account of how these models work.
The team studies the leading terms of the gradient, the biggest push in each training step. Those terms, they argue, explain how word links get written into the model. A mechanical story of learning ties deep learning back to language theory and may guide better training.
Catalog record
| CREATOR | Shawn Im, Changdae Oh, Zhen Fang, Sharon Li |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 101 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-19208
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
How Do Transformers Learn to Associate Tokens: Gradient Leading Terms Bring Mechanistic Interpretability Shawn Im, Changdae Oh, Zhen Fang, Sharon Li Abstract Semantic associations such as the link between “bird” and “flew” are foundational for language modeling as they enable models to go beyond memorization and instead generalize and generate coherent text. Understanding how these associations are learned and represented in language models is essential for connecting deep learning with linguistic theory and developing a mechanistic foundation for large language models. In this work, we analyze how these associations emerge from natural language data in attention-based language models through the lens of training dynamics. By leveraging a leading-term approximation of the gradients, we develop closed-form expressions for the weights at early stages of training that explain how semantic associations first take shape. Through our analysis, we reveal that each set of weights of the transformer has closed-form expressions as simple compositions of three basis functions–bigram, token-interchangeability, and context mappings–reflecting the statistics of the text corpus and uncovering how each component of the transformer captures semantic associations based on these compositions. Experiments on real-world LLMs demonstrate that our theoretical weight characterizations closely match the learned weights, and qualitative analyses further show how our theorem shines light on interpreting the learned associations in transformers. 1 Introduction Large language models (LLMs) based on self-attention have shown strong capabilities in capturing both factual knowledge and qualitative aspects of the human world ( 18 ; 45 ; 38 ; 1 ) . This progress has sparked growing interest in understanding why these models work so well and, in particular, what kinds of internal structures emerge during training ( 14 ; 27 ; 30 ; 8 ) . Among these structures, semantic associations are especially foundational to language modeling ( 19 ; 15 ; 31 ) , as they enable models to connect words and concepts in ways that support generalization and coherent text generation. While recent studies have identified specific mechanisms such as induction heads ( 35 ) , linear semantic relations ( 33 ) , and topic clustering ( 28 ) , we still lack a principled account of how semantic associations arise during the training of attention-based transformers . By semantic associations, we mean the statistical and functional relationships between tokens that encode meaning—for example, the link between “bird” and “flew”, the interchangeability of “car” and “truck” in adjectival contexts, or the coupling of “country” and “capital”. These associations have long been recognized in linguistics under the lens of distributional semantics ( 19 ) . In modern transformers, such associations are not explicitly programmed but instead emerge through gradient-based optimization over large corpora. Understanding how these structures crystallize during training is therefore essential not only for connecting deep learning with linguistic theory but also for developing a mechanistic foundation of representation learning in large language models. In this work, we develop a theory for the emergence of semantic associations in attention-based language models trained on natural language data, through the lens of training dynamics. A formal analysis of training dynamics is attractive as it allows us to rigorously discuss how modern language models learn features and capabilities. Unfortunately, the training dynamics of transformers are highly complex, which has led prior work to adopt unrealistic assumptions that diverge from practice: (1) synthetic structured language ( 28 ; 46 ) , (2) simplified model architectures without, e.g. , positional encoding or residual connections ( 39 ; 21 ) , and (3) non-standard training, such as sequential component-wise training or partially frozen weights ( 5 ; 28 ) . While these prior works provide valuable theoretical insights, their departures from realistic conditions raise concerns about the generalizability of their insights to LLMs used in practice. In contrast, we ground our study in a more realistic setting by focusing on naturalistic text distributions and attention-based transformers with positional encodings, optimized with a standard training procedure ( 6 ) . This is essential to minimize the gap between our theory and practical use. Our key technical innovation is to analyze training dynamics of the transformer at an early stage, through the leading term of an expansion of the gradients for each set of weights. In particular, transformers are known to acquire many core behaviors early in training–including semantic relations–and persist through convergence ( 35 ; 13 ; 33 ) . This makes the early phase not only empirically important but also analytically tractable. During this stage, gradient updates admit a closed-form approximation: the leading term dominates parameter updates before higher-order corrections accumulate. Leveraging this, we show that the learned weight matrices (including the output matrix, value matrices, query-key matrices) can be expressed as simple compositions of three basis functions: a bigram mapping , which captures next token dependencies; an interchangeability mapping , which reflects functional similarity across tokens (e.g., synonyms or shared grammatical roles); and a context mapping , which encodes longer-range prefix–suffix co-occurrence. Through experiments on a natural language dataset, we verify that the learned weights in an attention-based transformer model closely match our theoretical closed-form expressions, and further demonstrate that this holds even beyond the early stage. We also show rich qualitative examples of how each weight component of the transformer captures the actual word-wise semantic associations characterized by our theorem. Furthermore, we verify that our theoretically characterized features are correlated with the behavior of real-world language model. Figure 1 depicts an overview of our analysis, and we summarize our contributions as follows: Figure 1: To understand the emergence of associative features, we analyze the training dynamics of Transformers by focusing on the gradient leading terms for weights, which allows us to identify interpretable basis functions that characterize each weight by their compositions. Empirical validation confirms that our weight characterizations match the actual ones learned in practical transformers. 1. We present the first explicit characterization of weights in attention-based transformers trained on real-world text corpora under the next-token prediction loss; 2. We interpret the features learned in weights as compositions of bi-gram, interchangeability, and context mappings, and then show how these basis functions capture semantic association across words; 3. We finally validate our theoretical interpretation on both self-attention models and practical LLM, demonstrating the generality and relevance of our theorems. 2 Related Works Understanding emergence of features in Transformers. Many works have considered the training dynamics of transformers under controlled settings to interpret their feature learning ( 39 ; 5 ; 34 ; 23 ) . A line of them investigates how low-level associative features, such as bigram structure ( 5 ) , cyclic structure ( 21 ) , and co-occurrence ( 39 ; 46 ) , are learned from data. There are also multiple works that analyze how high-level capabilities, such as chain-of-thought ( 24 ) , topic clustering ( 28 ; 22 ) , reasoning or memorization ( 47 ) , and in-context learning capability ( 34 ; 5 ; 41 ; 23 ; 11 ) , are obtained during training. Although insightful, they often assume structured or abstract language data ( 28 ; 34 ; 46 ) , unrealistic model architecture ( 39 ; 7 ; 40 ) , and adjusted training strategies far from practice ( 5 ; 23 ; 21 )