SPEX : Scaling Feature Interaction Explanations for LLMs

Justin Singh Kang, Landon Butler, Abhineet Agarwal, Yigit Efe Erginbas, Ramtin Pedarsani, Kannan Ramchandran, Bin Yu · 2025

Read SPEX : Scaling Feature Interaction Explanations for LLMs by Justin Singh Kang, Landon Butler, Abhineet Agarwal, Yigit Efe Erginbas, Ramtin Pedarsani, Kannan Ramchandran, Bin Yu (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2502-13870.txt.

Description

Justin Singh Kang and co-authors presented SPEX in 2025, a method to explain large language model outputs by the interactions between input features. Popular explanation tools score each feature alone, and their extensions to interactions only work for short inputs. Real tasks often need much longer inputs, where features act together.

SPEX assumes that important interactions are sparse in real data. It uses a sparse Fourier method, borrowed from channel decoding, to find those interactions efficiently even for inputs of about a thousand features. On three hard long-context tasks, it rebuilt model outputs more faithfully than single-feature methods. On a question-answering set, its interactions matched human notes. It also explained closed and vision-language models.

Catalog record

CREATORJustin Singh Kang, Landon Butler, Abhineet Agarwal, Yigit Efe Erginbas, Ramtin Pedarsani, Kannan Ramchandran, Bin Yu
YEAR2025
COLLECTIONPAPERS
FORMATTXT
FILE SIZE73 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2502-13870

Browse the full library · About the Tower of Babel