Select or Project? Evaluating Lower-dimensional Vectors for LLM Training Data Explanations
Lukas Hinterleitner, Loris Schoenegger, Benjamin Roth · 2026
Read Select or Project? Evaluating Lower-dimensional Vectors for LLM Training Data Explanations by Lukas Hinterleitner, Loris Schoenegger, Benjamin Roth (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-16651.txt.
Introduction
Lukas Hinterleitner, Loris Schoenegger, and Benjamin Roth ask a plain question about large language models. When a model gives an answer, which training texts shaped that answer? Their 2026 preprint tests ways to trace such links. The usual tools rely on model gradients, which are huge in size.
To cut the cost, most teams use only part of the model. That part is often picked with no clear reason. The authors check two routes in a strict way. One route picks a small set of weights. The other presses the full signal into fewer numbers. Their tests help teams pick a route with proof, not habit.
Catalog record
| CREATOR | Lukas Hinterleitner, Loris Schoenegger, Benjamin Roth |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 36 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-16651
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Select or Project? Evaluating Lower-dimensional Vectors for LLM Training Data Explanations Lukas Hinterleitner, Loris Schoenegger, Benjamin Roth Abstract Gradient-based methods for instance-based explanation for large language models (LLMs) are hindered by the immense dimensionality of model gradients. In practice, influence estimation is restricted to a subset of model parameters to make computation tractable, but this subset is often chosen ad hoc and rarely justified by systematic evaluation. This paper investigates if it is better to create low-dimensional representations by selecting a small, architecturally informed subset of model components or by projecting the full gradients into a lower-dimensional space. Using a novel benchmark, we show that a greedily selected subset of components captures the information about training data influence needed for a retrieval task more effectively than either the full gradient or random projection. We further find that this approach is more computationally efficient than random projection, demonstrating that targeted component selection is a practical strategy for making instance-based explanations of large models more computationally feasible. 1 Introduction Different modalities for explaining language model (LM) behavior in terms of causal influences have been proposed recently, including feature-based ( 2 ; 3 ) , or mechanistic explanations ( 19 ; 15 ) . While these approaches explain language model behavior in terms of inputs or architectural components, recent work has enabled the generation of instance-based explanations that provide insights grounded in the model’s training data ( 9 ; 21 ; 7 ; 6 ) . In this paper, we specifically focus on methods that identify influential training examples through the analysis of model gradients. By retrieving a set of examples that influenced a given model output, these methods enable model debugging and interpretability tasks ( 13 ; 21 ; 7 , e.g.,) . However, their application to modern large language models is challenging due to high computational costs: for a 1B-parameter model, a single gradient requires over 4 GB of memory. To make analysis tractable, it must often be restricted to a subset of model parameters ( 30 , e.g, the word-embedding layer:) . In existing work, this choice was often made ad hoc and rarely justified by systematic evaluation. In this paper, we address this issue by investigating whether it is better to select a sparse, structurally informed subset of model components or to project the entire high-dimensional gradient into a dense, lower-dimensional space. Specifically, we systematically compare two strategies for creating gradient surrogates: An architecture-aware greedy layer selection algorithm proposed in this work, that selects a set of layer components whose combined gradients are most informative. And random projection , a widely used, architecture-agnostic technique that produces a low-dimensional representation of the full gradient ( 12 ) . We evaluate using a novel benchmark for instance-based explanations , designed to test how well a given selection supports a retrieval task in which the original training example must be identified among alternative prompt–completion pairs generated using various strategies. Our main contributions are threefold: (1) We propose an efficient retrieval-based benchmark for instance-based explanations. (2) We conduct a principled comparison of architecture-aware selection versus geometry-preserving projection for attribution. (3) We demonstrate that our targeted selection strategy can result in more accurate attributions while also being more efficient. We find that a greedily selected subset of model components captures the required information about training data influence more accurately than either the full gradient or random projection. We also find that this targeted selection is more computationally efficient than random projection. Our central insight is that a carefully chosen subset can provide a clearer and more discriminative signal for certain tasks, making it a practical strategy for instance-based explanations of large models. 1 1 1 We make our code available at https://doi.org/10.5281/zenodo.18346666 . Figure 1: Overview of the evaluation setup for the paraphrased setting. The gradient of a paraphrased sample s_{p_{i}} is compared to gradients from its original counterpart s_{i} and other candidates (e.g., s_{j} ). The method succeeds at this retrieval task if the cosine similarity is highest for the original pair. The setup is identical for the generated setting. 2 Related Work Instance-based explanation and attribution Instance-based explanation aims to identify influential training examples that explain a model’s prediction. A dominant framework is leave one out influence (the effect of removing a given training example has on the prediction), which can be approximated with re-training free methods such as influence functions ( 13 ; 6 ) . Alternative approaches, such as TracInCP ( 21 ) , aim to approximate training data influence during the training process by comparing the test instance’s loss gradient to training gradients at a set of model checkpoints. The challenge of high-dimensional gradients These influence estimation methods involve comparisons of model gradients (and calculating inverse Hessian-vector products for influence functions), which remains computationally intractable for LLMs: recent applications therefore restrict computation to a subset of model parameters, such as the multilayer perceptron (MLP) layers ( 6 ) or LoRA layers ( 14 ) . Another common choice is to restrict influence estimation to the model’s final layers ( 23 ; 1 ; 6 ) . However, 30 find this to be suboptimal for LLMs due to a cancellation effect in deeper layers, and propose using early layer components instead, for example, the more stable word embedding gradients. An alternative line of work projects full model gradients to a lower-dimensional space where pairwise distances between examples are preserved ( 20 ; 29 ; 16 , e.g.,) . Our work systematically compares the utility of all architectural components and random projection to determine which strategy provides the most informative signal for retrieval. Evaluation of training data explanations 6 evaluate their approximation of influence functions by reporting agreement with the proximal Bergmann response function , whose approximation does not require re-training. However, as they note themselves, it remains unclear whether this is a useful measure of data influence. 1 evaluate by introducing facts with the fine-tuning data, that the model of interest did not know before, and then verify whether influence estimation methods correctly attribute importance to this fine-tuning set when queried about them. Such evaluation strategies require expensive additional training steps that are infeasible with LLMs. Our evaluation setup is most similar to 28 who use paraphrased data to test the performance of influence estimation methods like Data Shapley ( 4 ) and influence functions by identifying the original sources of rewritten text. Similarly, our evaluation setup is re-training free; we use it to systematically compare dimensionality reduction and gradient selection strategies for influence estimation. 3 Method: Gradient Similarity for Instance-based Explanations Instance-based explanation methods must be capable of distinguishing training examples that influence model output from those that do not. Popular methods do so based on model loss gradients. As we will outline in Section 3.2 we frame this requirement as a retrieval problem: to study the quality of different low-dimensional representations, our setup measures how well they allow one to identify a given test example among a set of distractors. We introduce a novel low-dimensional representation strategy in Section 3.3 and describe our setup for random projection in Section 3.4 . Our evaluation approach is (1) local , targeting decisions at the instance level (each training sample/instance processed