Grad-ELLM: Gradient-based Explanations for Decoder-only LLMs
Xin Huang, Antoni B. Chan · 2026
Read Grad-ELLM: Gradient-based Explanations for Decoder-only LLMs by Xin Huang, Antoni B. Chan (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-03089.txt.
Description
Xin Huang and Antoni B. Chan tackle a measurement problem in explaining language models: checking whether an attribution, a ranking of which input words mattered, is actually faithful. Standard soft-perturbation scores can be inflated simply because a method keeps more words during testing, which confuses attribution quality with word count. The authors propose corrected metrics that compare methods at the same expected share of retained words.
They also introduce Grad-ELLM, an attribution method for decoder-only models that merges gradient-based channel importance with attention-based token importance at each generation step. In tests with Llama and Mistral on classification and open generation, Grad-ELLM scored strongly on comprehensiveness-style faithfulness under the corrected metric. Under the sufficiency-style metric, no single method dominated, a nuance the new framework makes visible.
Catalog record
| CREATOR | Xin Huang, Antoni B. Chan |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 45 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-03089