SPEX : Scaling Feature Interaction Explanations for LLMs
Justin Singh Kang, Landon Butler, Abhineet Agarwal, Yigit Efe Erginbas, Ramtin Pedarsani, Kannan Ramchandran, Bin Yu · 2025
Read SPEX : Scaling Feature Interaction Explanations for LLMs by Justin Singh Kang, Landon Butler, Abhineet Agarwal, Yigit Efe Erginbas, Ramtin Pedarsani, Kannan Ramchandran, Bin Yu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-13870.txt.
Description
Justin Singh Kang and co-authors presented SPEX in 2025, a method to explain large language model outputs by the interactions between input features. Popular explanation tools score each feature alone, and their extensions to interactions only work for short inputs. Real tasks often need much longer inputs, where features act together.
SPEX assumes that important interactions are sparse in real data. It uses a sparse Fourier method, borrowed from channel decoding, to find those interactions efficiently even for inputs of about a thousand features. On three hard long-context tasks, it rebuilt model outputs more faithfully than single-feature methods. On a question-answering set, its interactions matched human notes. It also explained closed and vision-language models.
Catalog record
| CREATOR | Justin Singh Kang, Landon Butler, Abhineet Agarwal, Yigit Efe Erginbas, Ramtin Pedarsani, Kannan Ramchandran, Bin Yu |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 73 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2502-13870