The Generalization Ridge: Information Flow in Natural Language Generation
Ruidi Chang, Chunyuan Deng, Hanjie Chen · 2025
Read The Generalization Ridge: Information Flow in Natural Language Generation by Ruidi Chang, Chunyuan Deng, Hanjie Chen (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2507-05387.txt.
Introduction
Ruidi Chang and co-authors trace how language models handle useful facts in 2025. Transformer models lead the field at writing text, yet how they gather task facts layer by layer is murky. Past work hints that middle layers hold the most broadly useful form of the input. The team built a frame, called InfoRidge, to track that flow during training.
Across models and data sets, useful signal peaks in the middle layers. It then falls in the last layers, a shift the authors link to rote storage. They call that peak pattern the generalization ridge. The map may guide where to read or tune a model for best results.
Catalog record
| CREATOR | Ruidi Chang, Chunyuan Deng, Hanjie Chen |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 50 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2507-05387
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
The Generalization Ridge: Information Flow in Natural Language Generation Ruidi Chang, Chunyuan Deng, Hanjie Chen Abstract Transformer-based language models have achieved state-of-the-art performance in natural language generation (NLG) tasks, yet their internal mechanisms for synthesizing task-relevant information remain insufficiently understood. While prior studies suggest that intermediate layers often yield more generalizable representations than final layers, how this generalization ability emerges and propagates across layers during training remains unclear. To address this gap, we propose InfoRidge, an information-theoretic framework, to characterize how predictive information—the mutual information between hidden representations and target outputs—varies across depth. Estimating this quantity enables us to trace the flow of task-relevant information throughout the model during training. Our experiments across various models and datasets reveal a consistent non-monotonic trend: predictive information peaks in upper-middle layers—forming a generalization ridge —before declining in final layers, reflecting a transition between generalization and memorization. To further investigate this phenomenon, we introduce residual scaling coefficients—trainable scalar parameters applied to each residual block—which serve as functional probes for assessing the relative importance of individual transformer layers. These coefficients reveal that, under distribution shift, models downweight final layers and increasingly rely on ridge layers, highlighting their role in generalization. Together, these findings offer new insights into the internal mechanisms of transformers and underscore the critical role of intermediate layers in supporting generalization. 1 Introduction Transformer-based language models have achieved remarkable performance in natural language generation (NLG) tasks such as summarization, machine translation, and dialogue generation vaswani2017attention ; dong2022survey . Nevertheless, we lack a rigorous understanding of how these models acquire and synthesize task-relevant information during training. A growing body of research has shown that intermediate layers in deep neural networks often surpass final layers in terms of representational quality and generalization performance liu2019linguistic ; voita2019bottom ; ansuini2019intrinsic ; ahrens2023read ; uselis2025intermediate . In language models, intermediate layers often encode richer semantic and more robust features than final layers fan2024not ; jin2024exploring ; skean2025layer . However, questions still remain: In NLG, how does information evolve across layers during training, and how are different layers of the network functionally organized to support generalization versus memorization? To investigate these questions, we propose InfoRidge, an information-theoretic framework, to analyze information flow in language models. Building on matrix-based mutual information estimation giraldo2014measures , our approach quantifies how predictive signals transform across layers during training. We center our analysis on two complementary quantities. The first is the predictive information at each layer, defined as the mutual information I(Z_{\ell};Y) between the hidden representation Z_{\ell} at layer \ell and the next-token label Y , which reflects how much task-relevant information is preserved. The second is the incremental information gain , denoted as I(\Delta Z_{\ell};Y) , where \Delta Z_{\ell} is the residual changes between successive layer \ell and \ell-1 . This measures the additional predictive information introduced by each transformer block. Using these tools, we uncover a non-monotonic trend: predictive information rises through early and middle layers, peaks in the upper-middle, and then declines in later layers. We term this phenomenon the generalization ridge , indicating a structural division of labor across depth. Intermediate layers appear to specialize in encoding abstract, generalizable features, while later layers increasingly reorient representations toward task-specific alignment and memorization. By further analyzing incremental information gain, we find that certain middle layers introduce the largest increases in predictive information, marking them as key contributors to the emergence of the ridge. Such layer-wise specialization offers insight into how generalization and memorization are distributed within the network. To further validate this interpretation, we introduce residual scaling coefficients —learnable scalar parameters \beta_{\ell} applied to each residual block—while keeping all other model weights fixed, drawing inspiration from prior work on layer-wise adaptation liu2019self ; menghani2024laurel . A higher \beta_{\ell} value indicates greater reliance on the corresponding layer’s output during prediction. These coefficients act as functional probes, revealing how the model redistributes layer-wise importance under different data distribution. Under in-distribution training, deeper layers retain higher residual weights, reflecting the model’s reliance on memorized, task-specific features. When evaluated under distribution shift, models reduce reliance on late layers and increase reliance on the ridge, further supporting its role in generalization. To understand the formation of the generalization ridge, we analyze both attention patterns and model capacity. Attention flow analysis shows that ridge layers attend more directly to semantically informative tokens, suggesting that the ridge is not only a statistical peak but also functionally aligned with meaningful computation. In contrast, deeper layers increasingly focus on positional tokens, consistent with a shift toward memorization. Beyond functional behavior, we also investigate the conditions under which the ridge emerges. Our depth ablation results show that the ridge only emerges beyond a certain depth threshold. Below this threshold, predictive information increases monotonically, indicating that sufficient capacity is a prerequisite for generalization ridge to emerge. Contributions. Our work provides a unified perspective on how predictive information is structured across depth in transformer-based language models for natural language generation tasks: 1. We propose InfoRidge, an information-theoretic framework, to quantify both predictive information and incremental information gain across layers, offering a dynamic view of how task-relevant signals emerge and evolve during training. 2. Through this framework, we identify the generalization ridge —a mid-layer peak in predictive information—revealing a transition between generalization and memorization. 3. We propose residual scaling coefficients as functional probes for layer importance, uncovering a shift in reliance from final layers to intermediate layers under distribution shift, consistent with the ridge’s role in supporting generalization. These findings suggest that transformer depth reflects functional stratification: intermediate layers encode core abstractions for generalization, while final layers increasingly tailor representations for memorization—often at the cost of robustness. This insight informs the design of more interpretable and adaptable architectures under distribution shift. 2 Related Work Understanding how information is encoded and transformed across layers in deep models has been widely studied through probing classifiers alain2016understanding , attention flow analysis vig2019analyzing , and information-theoretic tools such as the information bottleneck shwartz2017opening , mutual information estimation goldfeld2019estimating , and matrix-based entropy methods giraldo2014measures . These methods provide various lenses to quantify representational capacity, abstraction, and invariance across layers. A growing body of research has shown that intermediate layers in deep networks often outperform final layers in terms of representational quality and task performance ansuini2019intrinsic ; yosinski2014transferable ; uselis2025intermediate ; ahrens2023read ; ando2023use . In language models, mid-depth layers tend to capture richer semantic or robust features than output layers liu2019linguistic ; voita2019bottom ; jin2024exploring ; fan2024not . These findings challenge the assumption that deeper layers always yield better representations, revealing instead a peak in representational utility at intermediate depths. This pattern holds across settings such as transfer learning yosinski2014transferable , continual learning ahrens2023read , and out-of-distribution generalization uselis2025intermediate . Recent work by Skean et al. skean2025layer proposes a unified framework that evaluates representation quality using entropy, curvature, and invariance. Collectively, these results highlight the importance of intermediate layers for abstraction and generalization, motivating our analysis of layer-wise information dynamics through an information-theoretic lens. Differently, we focus on information flow for next token generation, specifically examining mutual information between the last input token and the target label. Our goal is to understand how generalization and memorization dynamics evolve during training from information-theoretic perspective. 3 InfoRidge: Information Estimation Framework Figure