Learning Task Representations from In-Context Learning

Baturay Saglam Zhuoran Yang Dionysis Kalogerias Amin Karbasi Yale University { baturay.saglam , zhuoran.yang , dionysis.kalogerias , amin.karbasi }@yale.edu · 2025

Read Learning Task Representations from In-Context Learning by Baturay Saglam Zhuoran Yang Dionysis Kalogerias Amin Karbasi Yale University { baturay.saglam , zhuoran.yang , dionysis.kalogerias , amin.karbasi }@yale.edu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-05390.txt.

Introduction

Baturay Saglam and co-authors study how large language models store the task hidden in a prompt. Such models learn new tasks from a few examples in the prompt, with no retraining. The team builds a task vector, a single summary of the task, from the model's attention heads. Weights for the heads are tuned step by step so the vector keeps the task intact.

Older ways to build such vectors fail once the task leaves plain text. The team adds a test bed based on fitting curves to data. Their method pulls task facts from examples and works for both text and curve tasks. The results suggest one vector can carry a task across kinds of data, a step toward reading how models adapt.

Catalog record

CREATORBaturay Saglam Zhuoran Yang Dionysis Kalogerias Amin Karbasi Yale University { baturay.saglam , zhuoran.yang , dionysis.kalogerias , amin.karbasi }@yale.edu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE82 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-05390

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Learning Task Representations from In-Context Learning Baturay Saglam Zhuoran Yang Dionysis Kalogerias Amin Karbasi Yale University { baturay.saglam , zhuoran.yang , dionysis.kalogerias , amin.karbasi }@yale.edu Abstract Large language models (LLMs) have demonstrated remarkable proficiency in in-context learning (ICL), where models adapt to new tasks through example-based prompts without requiring parameter updates. However, understanding how tasks are internally encoded and generalized remains a challenge. To address some of the empirical and technical gaps in the literature, we introduce an automated formulation for encoding task information in ICL prompts as a function of attention heads within the transformer architecture. This approach computes a single task vector as a weighted sum of attention heads, with the weights optimized causally via gradient descent. Our findings show that existing methods fail to generalize effectively to modalities beyond text. In response, we also design a benchmark to evaluate whether a task vector can preserve task fidelity in functional regression tasks. The proposed method successfully extracts task-specific information from in-context demonstrations and excels in both text and regression tasks, demonstrating its generalizability across modalities. Moreover, ablation studies show that our method’s effectiveness stems from aligning the distribution of the last hidden state with that of an optimally performing in-context-learned model. 1 Introduction Large language models (LLMs) based on the transformer architecture [ 50 ] have seen dramatic improvements in recent years. A notable feature of these models, such as GPT-3 [ 3 ] , is their capability for in-context learning (ICL). This process involves the model receiving a prompt that includes demonstrations of a task in the form of input-output pairs. When presented with a new query input, the model can generate the appropriate output by extrapolating from the provided examples. For instance, after being prompted with a few examples, these models are capable of producing the antonyms of given input words. A concrete example is \underbrace{{\color[rgb]{0,0,1}\text{vanish}\rightarrow\text{appear},\text{ short}\rightarrow\text{tall},}}_{\text{examples}}\underbrace{{\color[rgb]{0,0,1}\text{increase}\rightarrow}}_{\text{query}}\underbrace{~{\color[rgb]{1,0,0}\text{decrease}}}_{\text{completion}}, (1) where the blue text is the prompt and the red text is the completion provided by the model. Not limited to linguistic tasks, transformers can also in-context learn a general class of functions \mathcal{F} [ 8 ] . Specifically, for any function f\in\mathcal{F} , the model is capable of approximating f(x_{\text{query}}) for a new query input x_{\text{query}} . This class may include linear or nonlinear relationships, which can be represented by various machine learning models such as linear regression, multi-layer ReLU networks, and decision trees. The ICL capability is particularly intriguing as it allows models to adapt to a wide range of downstream tasks on-the-fly—i.e., without requiring any parameter updates post-training [ 3 ] . This indicates that models can extract abstract task information (the relationship between input-output pairs) from the prompt and use it to generate accurate responses for query inputs. However, due to the complex nature of the transformer architecture, the computational mechanisms that facilitate how models internally extract and encode task information are not well understood. Task vectors have been identified as key mechanisms for encoding task information during ICL [ 48 , 25 , 11 , 29 ] . These vectors capture how models process prompts and embed the abstract in-context task as a numerical representation. While the effectiveness of these vectors is empirically well-supported, the precise computation of a task vector within the transformer architecture remains debated. Early findings suggest that layer or attention activations significantly influence ICL performance [ 11 , 29 , 48 ] . However, no consensus exists on the optimal way to conceptualize or compute these vectors, resulting in varied methodologies often restricted to a single modality (e.g., only text, not numeric functions). Notably, existing approaches often fail to generalize to functional regression tasks, consisting of a range of linear and nonlinear functions. Furthermore, they require adaptation to individual transformer architectures. For instance, the layers over which the task vector is computed must be explicitly identified for each model [ 25 , 48 , 11 ] . These limitations raise concerns about the generalizability and practical utility of existing task vector formulations. Consequently, in the pursuit of a more standardized and automated pipeline for task encodings, we hypothesize that: In-context learning tasks can be effectively represented as a weighted sum of all attention heads in a transformer, with the weights learned causally through gradient descent. 1.1 Contributions and Findings The key contributions and findings of our study are outlined as follows. An Automated Task Formulation Independent of Modality and Architecture We propose a framework that approximates the internal weights transformers implicitly use to represent ICL tasks. By optimizing these weights through causal gradient descent, the framework effectively captures the model’s inherent task representations, supporting our hypothesis. Unlike existing methods that require prior model analysis—such as manually selecting hidden layers to perturb—our approach is fully automated. Given any dataset and autoregressive model, it seamlessly computes the task formulation without model-specific adjustments. A New Benchmark for Regression Tasks While prior work has explored where in-context tasks are represented, most studies focus solely on language, with limited attention to functional regression. To evaluate task encodings in regression tasks, we introduce a new benchmark motivated by recent findings on the role of positional information in reasoning tasks [ 61 ] . This setup offers an intuitive framework for analyzing task encodings and ICL capabilities in regression. Performance Gains Our method performs effectively on the proposed regression benchmark and demonstrates significant advantages in language. Ablation studies indicate that this performance results from aligning the distribution of the last hidden state with that of an optimally in-context-learned model, providing insights into how task steering is achieved. 2 Related Work Here, we discuss studies that specifically focus on learning task representations in ICL. A more comprehensive review of related works is available in Appendix A . Initial studies on developing task representations for transformers are documented in [ 23 , 41 , 33 , 36 , 17 ] . These works proposed methods to create compositional task encodings through model perturbations in the parameter space: “soft” prompts, codebooks, and meta-mappings. Notably, the term task vectors was first introduced in [ 17 ] . In contrast, through the application of causal mediation analysis [ 37 , 51 , 30 , 57 , 9 ] , function vectors were discovered to exist inherently within the transformer architecture and to exhibit strong causal effects [ 48 ] . This finding aligns with research on RNNs, where it was demonstrated that RNN hidden states can be grouped based on task similarities [ 22 , 13 ] . Efforts to represent ICL tasks often derive a task vector from the layer activations associated with the dummy token following the input query [ 11 ] . This approach was later refined by averaging the layer activations of dummy tokens across a few-shot prompt and optimizing them [ 25 ] . However, these methods were primarily designed for language tasks, where input-output pairs are explicitly separated by specific tokens (e.g., “ \rightarrow ”). This limitation restricts their applicability to broader domains, such as regression tasks. Another approach leverages the first principal component of the difference in layer activations to guide the ICL task, resulting in In-Context Vectors (ICVs) [ 29 ] . Notably, attention heads have been argued to play a critical role

Read the complete text