A Comparative Study of Learning Paradigms in Large Language Models via Intrinsic Dimension

Saahith Janapati, Yangfeng Ji · 2024

Read A Comparative Study of Learning Paradigms in Large Language Models via Intrinsic Dimension by Saahith Janapati, Yangfeng Ji (2024) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2412-06245.txt.

Introduction

Saahith Janapati and Yangfeng Ji compare two ways that large language models learn tasks, in work published in 2024. One way, fine-tuning, retrains the model on examples and changes its inner weights. The other, in-context learning, leaves the weights alone and puts examples in the prompt. The pair ask how each method reshapes the model's inner picture of a task.

They measure that picture with intrinsic dimension, a count of how many degrees of freedom the model's states really use. Fine-tuning changed that count as training went on, and more prompt examples changed it too. In-context learning always gave a higher count than fine-tuning. The result suggests that prompt-based learning spreads its work across a wider space inside the model.

Catalog record

CREATORSaahith Janapati, Yangfeng Ji
YEAR2024
COLLECTIONPAPERS
FORMATTXT
FILE SIZE36 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2412-06245

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

A Comparative Study of Learning Paradigms in Large Language Models via Intrinsic Dimension Saahith Janapati, Yangfeng Ji Abstract The performance of Large Language Models (LLMs) on natural language tasks can be improved through both supervised fine-tuning (SFT) and in-context learning (ICL), which operate via distinct mechanisms. SFT updates the model’s weights by minimizing loss on training data, whereas ICL leverages task demonstrations embedded in the prompt, without changing the model’s parameters. This study investigates the effects of these learning paradigms on the hidden representations of LLMs using Intrinsic Dimension (ID). We use ID to estimate the number of degrees of freedom between representations extracted from LLMs as they perform specific natural language tasks. We first explore how the ID of LLM representations evolves during SFT and how it varies due to the number of demonstrations in ICL. We then compare the IDs induced by SFT and ICL and find that ICL consistently induces a higher ID compared to SFT, suggesting that representations generated during ICL reside in higher dimensional manifolds in the embedding space. 1 1 1 Code is available at the following GitHub repo . 1 Introduction Large Language Models (LLMs) have transformed the field of Natural Language Processing through their general natural language understanding capabilities, which can be applied to a broad range of tasks. The performance of an LLM on a specific task can be improved through two primary learning paradigms: supervised fine-tuning (SFT) and in-context learning (ICL). SFT adapts pre-trained models to specific tasks by updating their parameters, while ICL requires no parameter updates, relying instead on task-specific demonstrations within the model’s context window. Despite their widespread success, how these methods influence a model’s internal representation space is still not fully understood. Intrinsic dimension (ID) is a useful metric for assessing the geometric complexity of a model’s representations. It quantifies the number of degrees of freedom in the representation space, serving as a measure of the complexity of the underlying manifolds where the embeddings reside. In this work, we analyze the intrinsic dimension (ID) of hidden representations across model layers during task execution under both supervised fine-tuning (SFT) and in-context learning (ICL). Specifically, we explore: • How fine-tuning duration influences ID of representations on both training and validation data. • How the number of demonstrations used in ICL affects ID of representations. Our findings reveal that (1) the ID sometimes decreases during the early stages of fine-tuning but generally increases in the later stages, and (2) the ID increases initially with more demonstrations in ICL, then either plateaus or decreases as the number of demonstrations continues to rise. We then conduct experiments directly comparing the intrinsic dimensions of ICL and fine-tuning across several models and datasets. We find that the intrinsic dimensions of representations from fine-tuned models are generally lower than those from models using ICL, even though the fine-tuned models achieve higher accuracy than the ICL models. Additionally, our results suggest that ID may serve as a practical heuristic for selecting the optimal number of demonstrations in ICL to maximize performance while minimizing input length. These findings shed light on the differing impacts that the two learning paradigms have on the representation space of LLMs. Figure 1: Accuracy, intrinsic dimension, and normalized AUC for the Llama-3-8B model on the MMLU dataset. (a) Fine-tuning achieves the highest accuracy. (b) ICL produces intermediate representations with higher intrinsic dimensions across model layers compared to zero-shot (ICL-0) and fine-tuned models. (c) Normalized AUC increases with the number of demonstrations in ICL, while fine-tuned models exhibit lower AUC. 2 Background 2.1 Decoder Transformer Architecture LLMs are built on the Transformer decoder architecture, which processes token sequences through a series of Transformer layers. In each layer, token representations are updated via self-attention that considers only the preceding tokens from the previous layer, progressively encoding information for the next-token prediction task. The final layer then uses the representation of the last token to predict the next token in the sequence. In this work, we analyze the intrinsic dimension of the representations corresponding to the last token of sequences where LLMs are prompted to perform specific natural language tasks. 2.2 Intrinsic Dimension Estimation Intrinsic dimension (ID) refers to the minimal number of variables required to capture the essential structure of high-dimensional data. Although modern neural networks operate in high-dimensional spaces (e.g., the hidden representations of Llama-3-8B span 4096 dimensions), the representations corresponding to a specific dataset or task often lie on a manifold of much lower dimension. This occurs because the network disentangles and extracts the most relevant lower-dimensional features needed to complete the task. According to the manifold hypothesis, real-world data typically resides on a low-dimensional manifold ( Goodfellow 2016 ) . Therefore, to effectively solve tasks—such as next-token prediction—neural networks must learn representations that align with this low-dimensional structure. Consequently, the intrinsic dimension of data representations provides unique insight into the complexity of the representation spaces constructed across the layers of a neural network. In this work, we estimate the intrinsic dimension (ID) of our representations using the TwoNN estimator , as introduced by Facco et al. 2017 . We chose this method because of its simplicity, computational efficiency, and robustness when handling datasets with non-uniform densities and high-dimensional curvature—common challenges in neural network representations. The TwoNN estimator operates on a set of points by computing the distances to each point’s first ( r_{1} ) and second ( r_{2} ) nearest neighbors. For a given point x , the ratio \mu=\frac{r_{2}}{r_{1}} is calculated. The intrinsic dimension d is then derived from the empirical cumulative distribution function (CDF) of \mu . Specifically, the log-linear relationship between \log(\mu) and \log\bigl(1-F_{\text{emp}}(\mu)\bigr) , where F_{\text{emp}}(\mu) is the empirical CDF, is used to estimate d : d=-\frac{\log\bigl(1-F_{\text{emp}}(\mu)\bigr)}{\log(\mu)} The TwoNN estimator has been successfully applied in several prior works analyzing the intrinsic dimension of neural network representations, including Sharma and Kaplan 2022 , Ansuini et al. 2019 , Valeriani et al. 2024 , and specifically in large language models (LLMs) by Cheng et al. 2023 and Lee et al. 2024 . We also validate the correlation between the TwoNN estimator and another widely used intrinsic dimension estimator—the Maximum Likelihood Estimator introduced by Levina and Bickel 2004 —in Appendix F as a sanity check. 3 Related Works 3.1 Supervised Fine-Tuning in LLMs Pre-trained LLMs can be quickly adapted to improve performance on natural language tasks through supervised fine-tuning, which updates the model’s parameters via gradient descent on task-specific training examples. Aghajanyan et al. 2020 show that fine-tuning large language models often requires updating only a low-dimensional subspace of parameters to achieve near-optimal performance. (Note that their work focuses on the intrinsic dimension of the parameter space, whereas our work examines the intrinsic dimension of the representation space.) Building on this, Hu et al. 2021 introduce Low-Rank Adaptation (LoRA), a method that injects low-rank matrices into the weight matrices for fine-tuning instead of updating all parameters. We employ LoRA for all our fine-tuning experiments. 3.2 In-Context Learning Introduced in GPT-3 by Brown 2020 , ICL (or few-shot learning) refers to the ability of LLMs to learn to perform a task in a single forward pass, using (input, output) pairs embedded in a prompt. Dai et al. 2022 provides evidence

Read the complete text