ExplainerPFN: Towards tabular foundation models for model-free zero-shot feature importance estimations
Joao Fonseca, Julia Stoyanovich · 2026
Read ExplainerPFN: Towards tabular foundation models for model-free zero-shot feature importance estimations by Joao Fonseca, Julia Stoyanovich (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-23068.txt.
Introduction
Joao Fonseca and Julia Stoyanovich estimate feature importance without opening the model. Knowing which inputs drive a classifier's decisions is key to explaining it. Shapley values are a popular answer, but they need direct model access and can be too costly to compute exactly.
Their ExplainerPFN approach aims at model-free, zero-shot importance scores for tabular data. Foundation-model style design lets it answer for new datasets at once, without retraining. Cheap, access-free explanations help audit models that vendors keep closed. Zero-shot estimation matters because a fresh dataset should not need a fresh explanation model trained before anyone can use it.
Catalog record
| CREATOR | Joao Fonseca, Julia Stoyanovich |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 70 KB |
| VERSION | 1.0 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2601-23068
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
ExplainerPFN: Towards tabular foundation models for model-free zero-shot feature importance estimations Joao Fonseca, Julia Stoyanovich Abstract Computing the importance of features in supervised classification tasks is critical for model interpretability. Shapley values are a widely used approach for explaining model predictions, but require direct access to the underlying model, an assumption frequently violated in real-world deployments. Further, even when model access is possible, their exact computation may be prohibitively expensive. We investigate whether meaningful Shapley value estimations can be obtained in a zero-shot setting, using only the input data distribution and no evaluations of the target model. To this end, we introduce ExplainerPFN, a tabular foundation model built on TabPFN that is pretrained on synthetic datasets generated from random structural causal models and supervised using exact or near-exact Shapley values. Once trained, ExplainerPFN predicts feature attributions for unseen tabular datasets without model access, gradients, or example explanations. Our contributions are fourfold: (1) we show that few-shot learning-based explanations can achieve high fidelity to SHAP values with as few as two reference observations; (2) we propose ExplainerPFN, the first zero-shot method for estimating Shapley values without access to the underlying model or reference explanations; (3) we provide an open-source implementation of ExplainerPFN, including the full training pipeline and synthetic data generator; and (4) through extensive experiments on real and synthetic datasets, we show that ExplainerPFN achieves performance competitive with few-shot surrogate explainers that rely on 2--10 SHAP examples 1 1 1 We make all the code developed for this project available at https://github.com/joaopfonseca/ExplainerPFN . 1 Introduction In many real-world applications, machine learning (ML) systems make or assist in making high-stakes decisions, such as loan approvals, hiring, or medical diagnoses. These models are often proprietary and operate as black boxes, leaving users and regulators without insight into how decisions are made. Consider an individual who is denied a loan by a proprietary, black-box ML system. The applicant may reasonably ask which factors contributed most to the decision. Alternatively, stakeholders may wish to ensure that the model does not distinguish based on protected attributes like race or gender. However, access to the underlying model is rarely granted, and feature-level explanations are almost never provided. In such settings, the ability to estimate feature attributions without model access becomes crucial for auditing decisions, identifying potential biases, and improving transparency. This challenge reflects a broader trend in modern ML. Many real-world models operate behind closed APIs for reasons of privacy, intellectual property, or security, limiting direct model access. At the same time, tabular data has received comparatively little attention in foundation model research. Recent work highlights both the need for methods that can analyze data without direct access to sensitive models ( 13 ) and the potential of large tabular models to transfer knowledge across heterogeneous tabular tasks ( 39 ) . Recent studies raise questions about the stability and interpretability of existing feature attribution methods. While covariate-based importance measures are statistically well motivated in many settings ( 41 ) , Shapley-based explanations can be sensitive to correlations among input features, and even simple changes in data representation can substantially alter SHAP-based attributions ( 17 ) . Rather than being a purely methodological limitation, these sensitivities suggest that the data distribution itself plays a central role in shaping feature attributions. This observation motivates the possibility of learning attribution patterns directly from data, without requiring access to the underlying model. Contributions. We study zero-shot estimation of Shapley value feature attributions using only the input data distribution and model predictions. We approach this problem through tabular foundation models and in-context learning, and ask whether Shapley values can be estimated in highly resource-constrained settings without access to the model, gradients, or reference SHAP explanations. We introduce ExplainerPFN, a tabular foundation model built on top of TabPFN ( 14 ) that predicts Shapley values in a fully zero-shot manner. Our main contributions are: 1 We show that few-shot learning-based explanations can achieve high fidelity to true SHAP values, in some cases with as few as 2 reference observations. 2 We propose the first zero-shot method for estimating Shapley values, requiring no access to the underlying ML model or feature attribution examples. 3 We provide an open-source implementation of ExplainerPFN, including the training pipeline, pretrained weights, and a synthetic data generator for tabular foundation model pretraining, adapted from the TabPFN synthetic prior-fitting procedure and redesigned for feature attribution learning. 4 Through extensive experiments on real and synthetic datasets, we demonstrate that ExplainerPFN achieves performance competitive with few-shot surrogate explainers requiring 2–10 SHAP examples from the target model. Overall, our results suggest that meaningful feature attribution patterns can be recovered directly from the data distribution, opening a new direction in zero-shot explainability using tabular foundation models. 2 Preliminaries and Background Shapley values. The Shapley value is a concept from cooperative game theory that provides a fair way to distribute the total gains (or costs) among the players based on their individual contributions to the overall outcome ( 31 ) . It has been widely adopted in explainable AI (xAI) to attribute feature importance in ML models. Let f:\mathbb{R}^{m}\to\mathbb{R} be a predictive model, x_{i}=(x_{i}^{1},\ldots,x_{i}^{m}) an input instance 2 2 2 We use “input instance” and “observation” interchangeably. , and M=\{1,\ldots,m\} the set of feature indices. Given a dataset X=\{x_{1},\ldots,x_{n}\} , where x_{i}^{j} denotes the value of the j -ith feature for instance i , the Shapley value \phi_{i}^{j} is computed as follows: \Delta_{j}(S)\coloneqq\mathbb{E}\left[f(x_{i})\mid x_{i}^{S\cup\{j\}}\right]-\mathbb{E}\left[f(x_{i})\mid x_{i}^{S}\right] (1) w(S)\coloneqq\frac{|S|!(m-|S|-1)!}{m!} (2) \phi_{i}^{j}\coloneqq\sum_{S\subseteq M\setminus\{j\}}w(S)\Delta_{j}(S) (3) where S is a subset of features not including feature j , \mathbb{E}\left[f(x_{i})\mid x_{i}^{S}\right] is the expected model output conditioned on the feature values in S , and |S| is the cardinality of S . Intuitively, \Delta_{j}(S) represents the marginal contribution of feature j , which is weighted by w(S) to account for all feature coalitions. Finally, \phi_{i}^{j} is obtained by summing the weighted marginal contributions over all subsets S . SHAP. SHapley Additive exPlanations (SHAP) ( 25 ) is a popular game-theoretic framework for local explanations of ML model predictions in the form of feature importance scores derived from Shapley values. For general models (including neural networks), the exact computation of SHAP scores is intractable ( \#P -hard) ( 2 ) . Consequently, SHAP (and other Shapley value-based frameworks) typically approximate feature importance either via Monte Carlo sampling over S , by limiting the maximum coalition size ( 29 ) , or via model-specific approximations such as TreeSHAP ( 24 ) . These strategies introduce a trade-off between computational cost and fidelity of the explanation, defined by how well the resulting feature importance scores reconstruct the model output. Recent works aim to improve the computational efficiency of SHAP computation in particular settings ( 27 ; 2 ) , typically in polynomial time. Importantly, these methods still require access to the underlying model f . Instead, we consider the problem of estimating feature importance when f is not directly accessible. We assume access to X and the corresponding model predictions \hat{Y}\coloneqq{\hat{y}_{1},\ldots,\hat{y}_{n}} , where \hat{y}_{i}\coloneqq f(x_{i}) , and our objective is to approximate \phi_{i}^{j} . Tabular foundation models (TFM). Formally, a TFM is a pretrained mapping F\colon\mathcal{X}\times\mathcal{D}\to\mathbb{R} , where \mathcal{X}\subseteq\mathbb{R}^{m} denotes the feature space and \mathcal{D} denotes the space of possible context datasets,