Gradient Atoms: Unsupervised Discovery, Attribution and Steering of Model Behaviors via Sparse Decomposition of Training Gradients
J Rosser · 2026
Read Gradient Atoms: Unsupervised Discovery, Attribution and Steering of Model Behaviors via Sparse Decomposition of Training Gradients by J Rosser (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2603-14665.txt.
Introduction
Rosser finds model behaviors without labels by splitting training gradients into sparse atoms. Attribution tools need a preset behavior query, then score each document. Broad concepts shared across documents slip past that design. Gradient Atoms breaks per-document gradients into dictionary parts in a preconditioned eigenspace. Among 500 atoms, the tightest ones match refusal, arithmetic and trivia skills. The atoms also steer models as weight shifts. Bulleted-list output jumps from 33% to 94% under one atom, Rosser reports.
Catalog record
| CREATOR | J Rosser |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 24322 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2603.14665
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Gradient Atoms: Unsupervised Discovery, Attribution and Steering of Model Behaviors via Sparse Decomposition of Training Gradients J Rosser Abstract Training data attribution (TDA) methods ask which training documents are responsible for a model behavior. However, models often learn broad concepts shared across many examples. Moreover, existing TDA methods are supervised—they require a predefined query behavior, then score every training document against it—making them both expensive and unable to surface behaviors the user did not think to ask about. We present Gradient Atoms , an unsupervised method that decomposes per-document training gradients into sparse components (“atoms”) via dictionary learning in a preconditioned eigenspace. Each atom captures a shared update direction induced by a cluster of functionally similar documents, directly recovering the collective structure that per-document methods do not address. Among 500 discovered atoms, the highest-coherence ones recover interpretable task-type behaviors—refusal, arithmetic, yes/no classification, trivia QA—without any behavioral labels. These atoms double as effective steering vectors: applying them as weight-space perturbations produces large, controllable shifts in model behavior (e.g., bulleted-list generation 33% → \to 94%; systematic refusal 50% → \to 0%). The method requires no query–document scoring stage, and scales independently of the number of query behaviors of interest. Code is available at https://github.com/jrosseruk/gradient_atoms . 1 Introduction Figure 1: Gradient atoms discovered via sparse dictionary learning over 5,000 training-document gradients. Each point is one atom; high-coherence atoms correspond to tightly defined task types. Training data attribution (TDA) attributes a model’s behavior to the documents it was trained on ( 5 ; 3 ; 1 ) . Per-document attribution has proven valuable for identifying influential examples, debugging predictions, and curating datasets. However, per-document scoring captures only part of what training teaches. A model that learns to perform arithmetic during fine-tuning does so not because of any single arithmetic example, but because many arithmetic examples induce a shared update direction across weights. More broadly, 8 show that fine-tuning primarily instils procedural capabilities—task-level strategies such as classification, editing, or code generation—that emerge from the collective gradient signal of functionally similar documents. Standard TDA also faces a practical bottleneck: it is supervised , requiring the user to specify a query behavior before scoring documents against it. This demands (1) knowing what behaviors to look for in advance, and (2) an O(N) scoring pass per query. For influence-function methods like EKFAC ( 3 ; 1 ) , attributing Q behaviors costs O(Q\times N) query–document comparisons. For a comprehensive audit of learned behaviors, this is prohibitively expensive. We address both gaps with a different question. Instead of asking “which document caused this behavior?”, we ask: what are the shared update directions that clusters of documents jointly induce? This is the idea behind Gradient Atoms . We extract per-document gradients, project them into a preconditioned eigenspace (via EKFAC), and apply sparse dictionary learning to decompose them into atoms. Each atom is a direction in weight space shared by a cluster of functionally similar documents. The method is: • Unsupervised : no behavioral labels, measurement functions, or contrastive pairs are needed. • Efficient : one decomposition surfaces candidate behaviors simultaneously, with no per-query scoring stage. • Actionable : each atom can be unprojected into a full-parameter steering vector. We validate this in two stages. First, we show that sparse dictionary learning on EKFAC-preconditioned gradients discovers 500 atoms, of which the highest-coherence atoms correspond to interpretable task types ( Table 4 ). Second, we show that these atoms—discovered without any behavioral labels—function as effective steering vectors : applying them as weight perturbations produces large, controllable shifts in five tested behaviors ( Figure 2 ). Our contributions are: 1. We show that per-document TDA provides an incomplete account of fine-tuning and propose a richer unit of analysis: shared update directions in gradient space. 2. We introduce Gradient Atoms, an unsupervised method that discovers candidate model behaviors from training gradients alone, without supervision or per-query scoring. 3. We demonstrate that discovered atoms function as effective steering vectors, producing large controllable shifts in model behavior without any behavioral labels. 2 Related Work Several concurrent works share our core insight that gradient similarity reflects functional similarity among training examples. GradientSpace ( 11 ) clusters LoRA gradients to identify “latent skills” but uses them to train specialised expert routers, not for interpretability or steering. Mode-Conditioning ( 13 ) confirms that gradient clustering reliably recovers functional groupings (98.7% F1), but applies this to test-time compute allocation. ELROND ( 10 ) is philosophically closest: they decompose per-sample gradients into steerable directions via PCA and sparse autoencoders, but in diffusion models rather than LLMs. Our contribution combines: (1) operating on training gradients rather than inference activations, (2) being fully unsupervised with no per-query scoring, and (3) the discovered atoms directly functioning as steering vectors for LLMs. Standard TDA methods ( 5 ; 3 ; 1 ) require a predefined query and O(Q\times N) query–document comparisons; atoms complement this with unsupervised discovery. In the activation-space literature, SAEs ( 6 ) and gradient-informed variants ( 7 ; 9 ) decompose activations at inference time; atoms decompose what the model learned during training, across all layers simultaneously. Extended discussion is in Appendix A . 3 Method Consider a language model with parameters \theta fine-tuned on a dataset of N input–output pairs \{x_{1},\ldots,x_{N}\} . Each document x_{i} induces a gradient g_{i}=\nabla_{\theta}\mathcal{L}_{\text{CE}}(\theta;x_{i})\in\mathbb{R}^{d} , where d is the number of trainable parameters. The gradient g_{i} is the direction the model’s weights would move to improve on document i . Gradient Atoms decomposes these per-document gradients into sparse components; the highest-quality components isolate interpretable learned behaviors. The pipeline has five steps. 3.1 Per-Document Gradient Extraction For each training document x_{i} , we compute the gradient of cross-entropy loss with respect to all trainable parameters, producing a gradient matrix G\in\mathbb{R}^{N\times d} . Documents that require similar computations tend to produce similar gradient vectors. 3.2 EKFAC Projection and Preconditioning The raw gradient space is anisotropic—some directions have high curvature (small weight changes cause large loss changes) and others have low curvature. Without correction, any decomposition is dominated by high-curvature directions, drowning out semantic structure. We use the EKFAC eigendecomposition ( 3 ; 1 ) of the approximate Fisher information matrix to correct for this. For each module m with eigenvectors Q_{m} and eigenvalues \lambda_{m} , we project each gradient into the top- k eigenvectors and precondition: \tilde{g}_{i}^{(m)}=Q_{m}^{(k)\top}\,g_{i}^{(m)},\qquad\hat{g}_{i}^{(m)}=\frac{\tilde{g}_{i}^{(m)}}{\sqrt{\lambda_{m}^{(k)}+\epsilon}} (1) After concatenating across all M modules, the projected gradient is \hat{g}_{i}\in\mathbb{R}^{k_{\text{total}}} where k_{\text{total}}=k\times M . This makes the space approximately isotropic: a unit step in any direction corresponds to a roughly equal change in loss, encouraging atoms to capture functionally distinct directions rather than curvature artifacts. 3.3 Sparse Dictionary Learning We normalize each projected gradient to unit norm (so atoms reflect direction, not magnitude) and apply sparse dictionary learning to decompose: \hat{g}_{i}\approx\sum_{j=1}^{K}\alpha_{ij}\,d_{j} (2) where D=[d_{1},\ldots,d_{K}]\in\mathbb{R}^{K\times k_{\text{total}}} are the atoms and \alpha_{ij} are sparse coefficients—most are zero, so each document is explained by a few atoms. The sparsity penalty encourages each atom to capture a single pattern rather than blending multiple unrelated behaviors. 3.4 Coherence Scoring For each atom j , we identify its activating documents—those with non-zero coefficient \alpha_{ij} —and compute a coherence score over the top- n activating documents \mathcal{S}_{j} : \text{coherence}(j)=\frac{1}{|\mathcal{S}_{j}|(|\mathcal{S}_{j}|-1)}\sum_{a\neq b\in\mathcal{S}_{j}}\cos(g_{a},g_{b}) (3) where g_{a},g_{b} are the raw