From Compression to Expression: A Layerwise Analysis of In-Context Learning

Jiachen Jiang, Yuxin Dong, Jinxin Zhou & Zhihui Zhu · 2025

Read From Compression to Expression: A Layerwise Analysis of In-Context Learning by Jiachen Jiang, Yuxin Dong, Jinxin Zhou & Zhihui Zhu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-17322.txt.

Introduction

Language models learn new tasks from examples in the prompt, and Jiachen Jiang, Yuxin Dong, Jinxin Zhou, and Zhihui Zhu ask where inside the model that learning happens, in this 2025 paper. They study the model's inner patterns layer by layer with tools from statistics and geometry. A clear pattern emerged, which they call layerwise compression-expression.

Early layers squeeze the examples into a compact code that captures the task. Later layers then expand that code, mixing it with the new question to form an answer. The pattern held across many tasks and model designs. It also grows stronger with bigger models and more examples, and it helps explain why the skill survives a few bad examples. The authors add a theory of how attention lowers both error kinds as examples pile up.

Catalog record

CREATORJiachen Jiang, Yuxin Dong, Jinxin Zhou & Zhihui Zhu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE61 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2505-17322

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

From Compression to Expression: A Layerwise Analysis of In-Context Learning Jiachen Jiang, Yuxin Dong, Jinxin Zhou & Zhihui Zhu Abstract In-context learning (ICL) enables large language models (LLMs) to adapt to new tasks without weight updates by learning from demonstration sequences. While ICL shows strong empirical performance, its internal representational mechanisms are not yet well understood. In this work, we conduct a statistical geometric analysis of ICL representations to investigate how task-specific information is captured across layers. Our analysis reveals an intriguing phenomenon, which we term Layerwise Compression-Expression : early layers progressively produce compact and discriminative representations that encode task information from the input demonstrations, while later layers express these representations to incorporate the query and generate the prediction. This phenomenon is observed consistently across diverse tasks and a range of contemporary LLM architectures. We demonstrate that it has important implications for ICL performance—improving with model size and the number of demonstrations—and for robustness in the presence of noisy examples. To further understand the effect of the compact task representation, we propose a bias-variance decomposition and provide a theoretical analysis showing how attention mechanisms contribute to reducing both variance and bias, thereby enhancing performance as the number of demonstrations increases. Our findings reveal an intriguing layerwise dynamic in ICL, highlight how structured representations emerge within LLMs, and showcase that analyzing internal representations can facilitate a deeper understanding of model behavior. 1 Introduction In-context learning (ICL) [ 1 , 2 ] has emerged as a powerful capability of large language models(LLMs), allowing them to perform new tasks by conditioning on a few input-output examples without weight updates. Despite being trained solely for next-token prediction, LLMs exhibit strong empirical performance across a wide range of NLP tasks through this mechanism. For example, pretrained LLMs can make correct predictions based on a sequence of input-separation-output pairs that encode semantic mappings. Given the same query token, the model can make different predictions based on the task defined by the demonstrations, such as d and C for the following two tasks, \displaystyle(\underbrace{a\to b,\ b\to c,}_{\text{Next Letter}}\ c\to?),\quad(\underbrace{a\to A,\ b\to B,}_{\text{To Uppercase}}\ c\to?) (1) Recent research has advanced several theoretical perspectives on explaining why ICL works—viewing ICL as posterior inference [ 3 ] , implicit meta-learning [ 4 ] , internal optimization [ 5 ] or mechanistic interpretations [ 6 ] . However, the underlying mechanism of how LLMs distinguish different tasks and use this information to guide their output remains unclear for ICL. To address this gap, we focus on the hierarchical feature learning across layers and formulate the following research question: How do LLMs extract and differentiate task information from shallow to deep layers during in-context learning? To investigate how task-specific information evolves across layers, we conduct a statistical geometric analysis of ICL representations across multiple tasks. Specifically, we consider a set of T ICL tasks, each associated with a distribution \mathcal{D}_{t} for input-output pairs. Then for each task t , we randomly sample K input-output pairs (also referred to as demonstrations ) from \mathcal{D}_{t} , which are combined with a query to form an ICL instance . We construct multiple such instances per task following this procedure. To quantify how the model compresses task information in its internal representations, we examine two key aspects: (1) how instances from the same task are clustered together, and (2) how instances from different tasks are distinguished from each other. Our analysis reveals an intriguing phenomenon, which we term the Layerwise Compression-Expression , summarized as: Layerwise Compression-Expression Phenomenon LLMs exhibiting ICL capabilities organize their layers into two parts with distinct behaviors: a compression part and an expression part. The early layers, comprising the compression part, progressively produce compact and discriminative representations that capture task information from the input demonstrations. The later layers, forming the expression part, apply these compact representations to the query to generate the output. Specifically, we introduce a metric called Task-Distance Normalized Variance (TDNV) 1 1 1 Following a similar conceptual framework to Class-Distance Normalized Variance (CDNV) [ 7 ] by viewing each task as a class. that measures the ratio of within-task variance to between-task distance: within-task variance indicates how well the representation from the same task are compressed, while between-task distance reflects the separation from different tasks. A lower TDNV indicates that representations of the same task samples are similar and representations of different task samples are distinguishable. Thus, TDNV serves as an effective method of how well the task information is compressed. By measuring TDNV across transformer layers, we can track how the model progressively encodes and transforms task information throughout its architecture. Figure 1 : Layer-wise compression to expression in ICL representations. TDNV first decreases then increases from shallow to deep layers, splitting the model into compression and expression stages. During the compression stage, task vector accuracy increases as task information is progressively extracted from demonstration pairs. During the expression stage, early-exit accuracy increases as output information is progressively decoded based on the input query. Refer to Appendix G for detailed explanation of task vector and early-exit accuracy. As shown in Figure 1 , TDNV first decreases and then increases from shallow to deep layers, splitting the model into compression and expression stages. To further support our hypothesis, we measure task vector accuracy [ 8 ] and early-exit accuracy [ 9 , 10 ] across layers to track task-specific and output-specific information. Task vector accuracy measures zero-shot ICL performance when injecting intermediate layer hidden states extracted under ICL settings. Early-exit accuracy measures performance by directly applying the final-layer classifier to intermediate hidden states. During compression, task vector accuracy increases while early-exit accuracy remains low, indicating that the model is compressing representations to encode task information. During expression, task vector accuracy decreases while early-exit accuracy rapidly increases, indicating that the model begins to incorporate query information and decode output-specific representations. As we show, this behavior has important implications for ICL performance and robustness. Contributions. Our main contributions can be summarized as follows: • By analyzing the hidden representations of ICL, we conceptualize and extensively examine the Layerwise Compression-Expression phenomenon. Our results show that it is prevalent across model architectures (transformer and state-space models) and task domains (symbolic, language understanding and multimodality), and emerges during the training process. • We show that larger models and more demonstrations lead to more compressed task representations, explaining why larger models and longer contexts yield better performance. To further understand the compressed representation, we propose a bias-variance decomposition and provide a theoretical analysis showing how attention mechanisms contribute to reducing both variance and bias, thereby enhancing performance as the number of demonstrations increases. • We show that noisy demonstrations result in less compressed representations and a corresponding drop in performance. However, the representations remain distinguishable with a certain amount of noise, which helps explain the robustness of ICL. Moreover, we find that errors in early demonstrations can be suppressed by later examples, and that errors in later demonstrations lead to less compressed representations than those in early ones. This highlights the recency effect [ 11 , 12 ] and the key role of later demonstrations. • Motivated by our analysis, we propose task-vector contrastive fine-tuning method to further compress task vectors and reduce TDNV. Fine-tuning GPT-2

Read the complete text