ProxySPEX: Inference-Efficient Interpretability via Sparse Feature Interactions in LLMs
Landon Butler, Abhineet Agarwal *, Justin Singh Kang *, Yigit Efe Erginbas, Bin Yu, Kannan Ramchandran · 2025
Read ProxySPEX: Inference-Efficient Interpretability via Sparse Feature Interactions in LLMs by Landon Butler, Abhineet Agarwal *, Justin Singh Kang *, Yigit Efe Erginbas, Bin Yu, Kannan Ramchandran (2025) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2505-17495.txt.
Introduction
Butler and co-authors propose in 2025 ProxySPEX, a faster way to find sparse feature interactions inside large language models. Models gain power from feature interplay, but listing all combinations scales badly. A recent method, SPEX, uses interaction sparsity yet needs tens of thousands of model runs.
The new method exploits hierarchy, since high order interactions usually arrive with their lower order subsets. Gradient boosted trees are fit to masked model outputs first, then key interactions are extracted. Across four hard high dimensional datasets, output reconstruction beat marginal methods by twenty percent. The method used ten times fewer model runs than SPEX and approximates Shapley values at scale.
Catalog record
| CREATOR | Landon Butler, Abhineet Agarwal *, Justin Singh Kang *, Yigit Efe Erginbas, Bin Yu, Kannan Ramchandran |
|---|---|
| YEAR | 2025 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 78 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2505-17495
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ProxySPEX: Inference-Efficient Interpretability via Sparse Feature Interactions in LLMs Landon Butler, Abhineet Agarwal *, Justin Singh Kang *, Yigit Efe Erginbas, Bin Yu, Kannan Ramchandran Abstract Large Language Models (LLMs) have achieved remarkable performance by capturing complex interactions between input features. To identify these interactions, most existing approaches require enumerating all possible combinations of features up to a given order, causing them to scale poorly with the number of inputs n n . Recently, Kang et al. (2025) proposed SPEX, an information-theoretic approach that uses interaction sparsity to scale to n ≈ 10 3 n\approx 10^{3} features. SPEX greatly improves upon prior methods but requires tens of thousands of model inferences, which can be prohibitive for large models. In this paper, we observe that LLM feature interactions are often hierarchical —higher-order interactions are accompanied by their lower-order subsets—which enables more efficient discovery. To exploit this hierarchy, we propose ProxySPEX , an interaction attribution algorithm that first fits gradient boosted trees to masked LLM outputs and then extracts the important interactions. Experiments across four challenging high-dimensional datasets show that ProxySPEX more faithfully reconstructs LLM outputs by 20% over marginal attribution approaches while using 10 × 10\times fewer inferences than SPEX. By accounting for interactions, ProxySPEX efficiently identifies the most influential features, providing a scalable approximation of their Shapley values. Further, we apply ProxySPEX to two interpretability tasks. Data attribution , where we identify interactions among CIFAR-10 training samples that influence test predictions, and mechanistic interpretability , where we uncover interactions between attention heads, both within and across layers, on a question-answering task. The ProxySPEX algorithm is available at https://github.com/mmschlk/shapiq . 1 Introduction Large language models (LLMs) have achieved great success in natural language processing by capturing complex interactions among input features. Modeling interactions is not only crucial for language, but also in domains such as computational biology, drug discovery and healthcare, which require reasoning over high-dimensional data. In high-stakes contexts, responsible decision-making based on model outputs requires interpretability. For example, in healthcare, a physician relying on LLM diagnostic assistance must intelligibly be able to explain their decision to a patient. Post-hoc feature explanation methods such as SHAP [ 1 ] and LIME [ 2 ] focus on marginal attributions and do not explicitly capture the effect of interactions. To address this limitation, recent work has proposed interaction indices, such as Faith-Shap [ 3 ] , that attribute all interactions up to a given order d by exhaustively enumerating them. With n features, enumerating O(n^{d}) interactions quickly becomes infeasible for even small n and d . Kang et al. [4] recently introduced SPEX, the first interaction attribution method capable of scaling up to n=1000 features. SPEX scales with n by observing that LLM outputs are driven by a small number of interactions. It exploits this sparsity by utilizing a sparse Fourier transform to efficiently search for influential interactions without enumeration. For example, with n=100 features, SPEX requires approximately 2\times 10^{4} model inferences to learn order 5 interactions—a small fraction of all possible 100^{5} interactions. Nonetheless, 2\times 10^{4} inferences is prohibitively expensive for large models. Hence, the question naturally arises: Can we identify additional structural properties among interactions to improve inference-efficiency? Figure 1 : ProxySPEX requires \sim 10\times fewer inferences to achieve equally faithful explanations as SPEX for a sentiment classification and image-captioning task using a BERT and CLIP model respectively. LASSO faithfulness plateaus indicating limits of marginal approaches. We show empirically that local (i.e., input specific) LLM feature interactions are often hierarchical : for an order d interaction, an LLM includes lower-order interactions involving subsets of those d features (see Figure 2 ). We use this to develop ProxySPEX , an interaction attribution algorithm that reduces the number of inferences compared to SPEX by 10\times while achieving equally faithful explanations. ProxySPEX exploits this local hierarchical structure by first fitting gradient boosted trees (GBTs) as a proxy model to predict the output of LLMs on masked input sequences. Then, ProxySPEX extracts important interactions from the fitted GBTs [ 5 ] . Evaluation overview. We compare ProxySPEX to marginal feature attributions and SPEX across four high-dimensional datasets with hundreds of features. Results are summarized below: 1. Faithfulness. ProxySPEX learns more faithful representations of LLM outputs than marginal approaches (\approx 15\%\text{ to }25\%) on average across datasets as we vary the number of inferences. Figure 1 compares explanation faithfulness of ProxySPEX to marginal attributions and SPEX. 2. Feature identification. By accounting for interactions, ProxySPEX identifies influential features that impact model outputs more significantly than marginal approaches, and can approximate Shapley values better than KernalSHAP in the low-inference regime. 3. Case study 1: Data attribution. Data Attribution is the problem of identifying training points responsible for a given test prediction. On CIFAR-10 [ 6 ] ProxySPEX identifies the interactions between training samples that most significantly impact classification performance. 4. Case study 2: Model component attribution. We use ProxySPEX to study interactions between attention heads, both within and across layers, on MMLU [ 7 ] for Llama-3.1-8B-Instruct [ 8 ] . We observe that intra-layer interactions become more significant for deeper layers. ProxySPEX identifies interactions that allow it to prune more heads than the LASSO. 2 Related work and applications Feature and interaction attribution. SHAP [ 1 ] and LIME [ 2 ] are widely used for model-agnostic feature attribution. SHAP uses the game-theoretic concept of Shapley values [ 9 ] for feature attribution, while LIME fits a sparse linear model [ 10 ] . Cohen-Wang et al. [11] also consider fitting a sparse linear model for feature attribution. Chen et al. [12] uses an information-theoretic approach for feature attributions. Other methods [ 13 , 14 ] study model structure to derive feature attributions. Sundararajan et al. [15] and Bordt and von Luxburg [16] define extensions to Shapley values that consider interactions. Fumagalli et al. [17] provides a framework for computing several interaction attribution scores, but their approach does not scale past n\approx 20 features, which prevents them from being applied to modern ML problems that often consist of hundreds of features. Note that some feature attribution approaches such as LIME and Faith SHAP [ 3 ] are formulated explicitly as a function approximation, while others are defined axiomatically such as SHAP, though one can typically construct equivalent function approximation objectives with a suitable distance metric. Fourier transforms and deep learning explainability. Several works theoretically study the spectral properties of transformers. Ren et al. [18] show transformers have sparse spectra and Hahn and Rofin [19] , Abbe et al. [20] establish that they are low degree. Abbe et al. [21] , Abbe et al. [22] study the bias of networks learning interactions via a “staircase” property, i.e., using lower-order terms to learn high-order interactions. Sparsity and low degree structure is also empirically studied in [ 23 , 24 ] . Kang et al. [25] shows that under sparsity in the Möbius basis [ 26 ] , a representation closely related to Shapley values and the Fourier transform, interaction attributions can be computed efficiently. Mohammadi et al. [27] also learn a sparse Möbius representation for computing Shapley values. Kang et al. [4] use these insights to propose SPEX, the first robust interaction attribution algorithm to scale to the