The Attribution Contract: Feature Attribution for Generative Language Models

Giang Nguyen · 2026

Read The Attribution Contract: Feature Attribution for Generative Language Models by Giang Nguyen (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-23080.txt.

Introduction

Nguyen argues that feature attribution scores carry no fixed meaning for generative models. A generative model emits tokens one by one, each token is both output and input, so explaining the output poses several questions. The Attribution Contract names the question a score set answers: the score, what is fixed, the target, the process, and the eligible features. Under a local next-token contract, generated tokens take 29 percent of attribution mass versus 25 percent for the input question. Mistaking that for a prompt-level explanation is the self-attribution fallacy.

Catalog record

CREATORGiang Nguyen
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE35403
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2605.23080

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

The Attribution Contract: Feature Attribution for Generative Language Models Giang Nguyen Abstract Feature attribution methods promise to identify which input features matter for a model output. In generative language models, however, it is often unclear what should count as a feature in the first place. In autoregressive language models, earlier generated tokens are both outputs of the model and inputs to later predictions. In diffusion language models, generation proceeds through iterative denoising or unmasking rather than fixed left-to-right prediction, so local explanation may target a state of diffusion rather than a next token. We argue that this ambiguity is not merely an implementation detail, but a conceptual limitation of carrying classifier-era feature attribution directly into generative language modeling. We introduce the Attribution Contract , a specification for feature-attribution claims that names what output is being explained, which features are eligible to receive attribution, what generative process is assumed, what is held fixed, and what model score is being attributed. The contract clarifies why the same attribution method can answer different questions depending on how it is instantiated. We argue that many disagreements about feature attribution in generative language models are not disagreements about attribution algorithms, but about unstated explanatory contracts. Using autoregressive and diffusion language models as case studies, we show when attribution to earlier generated tokens, intermediate states, or denoising stages is informative, when it is misleading, and why feature-attribution methods in generative language models should be evaluated as method–contract pairs. 1 Introduction Feature attribution methods are often described as answering a simple question: which input features mattered for this output? In the classical setting, this question has a natural shape. A model receives an input x , computes a prediction f(x) , and an attribution method assigns importance scores to components x_{i} relative to a scalar target such as a class logit, probability, or loss. Integrated Gradients [ 13 ] , for example, was introduced as an axiomatic method for attributing a deep network’s prediction to its input features, motivated by desiderata such as sensitivity and invariance. In generative language models (LMs), many real-world problems depend on feature attribution. A practitioner debugging a hallucinated summary needs to know whether the hallucinated content came from the source document or from the model’s own prior tokens [ 24 , 23 ] . A safety researcher auditing a harmful generation needs to know which part of the input triggered it [ 22 , 26 ] . A scientist studying in-context learning needs to know which prompt features the model is actually using to make predictions [ 25 ] . Figure 1: The same attribution method produces different attribution maps under different settings. All three rows use the same model, prompt, generation, and attribution method (Integrated Gradients [ 13 ] ). Only the setting differs. Top: a local next-token setting attributes the prediction of noir over both prompt and prefix tokens, and mass concentrates on the generated prefix Le chien est because it is predictive of the next token. Middle: a prompt-conditioned setting holds the generated prefix fixed (gray, hatched) and attributes only over prompt tokens. “Held fixed” means the prefix remains in the forward pass at its actual value but is removed from the attribution path, so mass that would have gone to the prefix is now distributed only among prompt tokens. The result concentrates on black , the prompt token most predictive of noir . Bottom: a span-level setting targets the full generated span Le chien est noir and attributes only over prompt tokens; the prefix is again held fixed (gray, hatched). Generative LMs break the clean setup of classical attribution. Consider an autoregressive LM with prompt x and generated output y_{1:T} : p(y_{1:T}\mid x)=\prod_{t=1}^{T}p(y_{t}\mid x,y_{<t}). (1) If we ask which tokens explain the next-token prediction y_{t} , the previously generated tokens y_{<t} are genuine inputs to the model and may rightly receive high attribution. But those same tokens are also outputs of the model. As a result, feature attribution depends on what exactly we are trying to explain. An attribution to earlier-generated tokens may be appropriate if the goal is to explain the model’s local prediction of the next token. The same attribution answers a different question if we instead ask how the prompt influenced the overall response y_{1:T} . We develop both questions in Section 4 and show that they correspond to different feature-attribution contracts. Unlike in classification settings, the interpretation of a feature-attribution score in generative LMs depends on the explanatory goal. Suppose a model is prompted: Translate to French: “The dog is black.” and generates: Le chien est noir. For the target token “noir,” the generated prefix “Le chien est” is predictive: it helps determine which token is likely to come next. But if the explanatory question is which input token accounts for the meaning of “noir,” the answer should point to “black,” not to the generated prefix . These are different feature-attribution questions. The issue is not that attribution to the generated prefix is wrong. Rather, the role of earlier generated tokens changes across explanatory settings, and feature-attribution methods for generative LMs often leave this distinction implicit [ 12 , 5 , 18 ] , echoing a long-standing concern that interpretability claims in machine learning are under-specified without explicit attention to their motivations and properties [ 28 ] . As such, the same attribution score may be interpreted as answering a different feature-attribution question from the one it actually supports. We call this kind of interpretive error the self-attribution fallacy : treating attribution to generated-prefix tokens as if it answered a prompt-level explanatory question, without specifying the contract under which that interpretation is justified. We argue that a feature-attribution score in a generative language model does not carry its own interpretation: that interpretation depends on the explanatory setting. We call such a setting an Attribution Contract , specifying what output is being explained, which features are eligible to receive attribution, what generative process is assumed, what is held fixed, and what model score is being attributed. Contributions. To summarize, this paper makes four contributions. First, we identify contract ambiguity as a distinctive failure mode of feature attribution in generative language models. Second, we introduce the Attribution Contract , specifying the target, eligible feature set, generative process, conditioning regime, and attributed score of any feature-attribution claim. Third, we identify the self-attribution fallacy , in which attribution to generated-prefix tokens is misread as prompt-level explanation. Fourth, we propose that feature-attribution methods in generative language models should be evaluated as method–contract pairs rather than as context-free attribution algorithms. 2 From classifier attribution to generative attribution Classical feature attribution inherits a static input-output schema: x\longrightarrow f(x). (2) The candidate features are usually components of the input, such as pixels or tokens. The attribution target is usually a scalar, such as a class score, loss, or logit. Even in this setting, attribution has been criticized for being sensitive to the explanatory task and evaluation criterion [ 19 , 20 , 21 ] . Recent unification work argues that feature attribution, data attribution, and component attribution share methodological structure, even though they are often studied in fragmented literatures [ 17 ] . Generative language models replace this static schema with a

Read the complete text